Human-Object Interaction (HOI) detection aims to localize human-object pairs
and comprehend their interactions. Recently, two-stage transformer-based
methods have demonstrated competitive performance. However, these methods
frequently focus on object appearance features and ignore global contextual
information. Besides, vision-language model CLIP which effectively aligns
visual and text embeddings has shown great potential in zero-shot HOI
detection. Based on the former facts, We introduce a novel HOI detector named
ISA-HOI, which extensively leverages knowledge from CLIP, aligning interactive
semantics between visual and textual features. We first extract global context
of image and local features of object to Improve interaction Features in images
(IF). On the other hand, we propose a Verb Semantic Improvement (VSI) module to
enhance textual features of verb labels via cross-modal fusion. Ultimately, our
method achieves competitive results on the HICO-DET and V-COCO benchmarks with
much fewer training epochs, and outperforms the state-of-the-art under
zero-shot settings.

ISA-HOI 是一种新颖的 HOI 探测器，它广泛利用了 CLIP 的知识，通过对视觉和文本特征之间的交互语义进行对齐，提取图像的全局上下文和物体的局部特征来改善图像中的交互特征，并且通过跨模态融合提高动词标签的文本特征，最终在较少的训练时期内在 HICO-DET 和 V-COCO 基准上取得了竞争性的结果，并在零样本设置下超越了最先进的方法。