Accurately detecting active objects undergoing state changes is essential for
comprehending human interactions and facilitating decision-making. The existing
methods for active object detection (AOD) primarily rely on visual appearance
of the objects within input, such as changes in size, shape and relationship
with hands. However, these visual changes can be subtle, posing challenges,
particularly in scenarios with multiple distracting no-change instances of the
same category. We observe that the state changes are often the result of an
interaction being performed upon the object, thus propose to use informed
priors about object related plausible interactions (including semantics and
visual appearance) to provide more reliable cues for AOD. Specifically, we
propose a knowledge aggregation procedure to integrate the aforementioned
informed priors into oracle queries within the teacher decoder, offering more
object affordance commonsense to locate the active object. To streamline the
inference process and reduce extra knowledge inputs, we propose a knowledge
distillation approach that encourages the student decoder to mimic the
detection capabilities of the teacher decoder using the oracle query by
replicating its predictions and attention. Our proposed framework achieves
state-of-the-art performance on four datasets, namely Ego4D, Epic-Kitchens,
MECCANO, and 100DOH, which demonstrates the effectiveness of our approach in
improving AOD.

提出了一种基于知识集成和知识蒸馏的活动物体检测方法，通过提供关于物体相关的可行交互信息来改善活动物体检测，实现了当前最先进的性能。