The interaction decoder utilized in prevalent Transformer-based HOI detectors
typically accepts pre-composed human-object pairs as inputs. Though achieving
remarkable performance, such paradigm lacks feasibility and cannot explore
novel combinations over entities during decoding. We present L OGIC HOI, a new
HOI detector that leverages neural-logic reasoning and Transformer to infer
feasible interactions between entities. Specifically, we modify the
self-attention mechanism in vanilla Transformer, enabling it to reason over the
<human, action, object> triplet and constitute novel interactions. Meanwhile,
such reasoning process is guided by two crucial properties for understanding
HOI: affordances (the potential actions an object can facilitate) and proxemics
(the spatial relations between humans and objects). We formulate these two
properties in first-order logic and ground them into continuous space to
constrain the learning process of our approach, leading to improved performance
and zero-shot generalization capabilities. We evaluate L OGIC HOI on V-COCO and
HICO-DET under both normal and zero-shot setups, achieving significant
improvements over existing methods.

使用神经逻辑推理和 Transformer 来推断实体之间的可行互动，通过改进 Transformer 的自注意机制，使用逻辑约束学习过程，提高性能和零样本泛化能力。

神经逻辑人 - 物体交互检测

Neural-Logic Human-Object Interaction Detection

Recently, the DETR framework has emerged as the dominant approach for
human--object interaction (HOI) research. In particular, two-stage
transformer-based HOI detectors are amongst the most performant and
training-efficient approaches. However, these often condition HOI
classification on object features that lack fine-grained contextual
information, eschewing pose and orientation information in favour of visual
cues about object identity and box extremities. This naturally hinders the
recognition of complex or ambiguous interactions. In this work, we study these
issues through visualisations and carefully designed experiments. Accordingly,
we investigate how best to re-introduce image features via cross-attention.
With an improved query design, extensive exploration of keys and values, and
box pair positional embeddings as spatial guidance, our model with enhanced
predicate visual context (PViC) outperforms state-of-the-art methods on the
HICO-DET and V-COCO benchmarks, while maintaining low training cost.

通过可视化和精心设计的实验，我们研究了 DETR 框架在人物 - 物体交互研究中的问题，并通过增强图像特征的方式，改进了模型的谓词视觉上下文，从而在 HICO-DET 和 V-COCO 基准上超过了最先进的方法，同时保持了较低的训练成本。