Abstract
Recently, the DETR framework has emerged as the dominant approach forhuman--object interaction (HOI) research. In particular, two-stagetransformer-based HOI detectors are amongst the most performant andtraining-efficient approaches. However, these often condition HOIclassification on object features that lack fine-grained contextualinformation, eschewing pose and orientation information in favour of visualcues about object identity and box extremities. This naturally hinders therecognition of complex or ambiguous interactions. In this work, we study theseissues through visualisations and carefully designed experiments. Accordingly,we investigate how best to re-introduce image features via cross-attention.With an improved query design, extensive exploration of keys and values, andbox pair positional embeddings as spatial guidance, our model with enhancedpredicate visual context (PViC) outperforms state-of-the-art methods on theHICO-DET and V-COCO benchmarks, while maintaining low training cost.