Human-object interaction (HOI) detection aims at detecting human-object pairs
and predicting their interactions. However, the complexity of human behavior
and the diverse contexts in which these interactions occur make it challenging.
Intuitively, human-centric visual cues, such as the involved participants, the
body language, and the surrounding environment, play crucial roles in shaping
these interactions. These cues are particularly vital in interpreting unseen
interactions. In this paper, we propose three prompts with VLM to generate
human-centric visual cues within an image from multiple perspectives of humans.
To capitalize on these rich Human-Centric Visual Cues, we propose a novel
approach named HCVC for HOI detection. Particularly, we develop a
transformer-based multimodal fusion module with multitower architecture to
integrate visual cue features into the instance and interaction decoders. Our
extensive experiments and analysis validate the efficacy of leveraging the
generated human-centric visual cues for HOI detection. Notably, the
experimental results indicate the superiority of the proposed model over the
existing state-of-the-art methods on two widely used datasets.

在这篇论文中，我们提出了三个用于生成图像中的人类中心视觉线索的提示，并结合了多人的多视角来生成丰富的人类中心视觉线索，以用于人物 - 物体交互检测，我们的实验证明了利用这些生成的人类中心视觉线索进行交互检测的有效性，而我们提出的模型在两个广泛使用的数据集上的实验结果表明其在现有最先进方法上的优越性。