Although we have witnessed significant progress in human-object interaction
(HOI) detection with increasingly high mAP (mean Average Precision), a single
mAP score is too concise to obtain an informative summary of a model's
performance and to understand why one approach is better than another. In this
paper, we introduce a diagnosis toolbox for analyzing the error sources of the
existing HOI detection models. We first conduct holistic investigations in the
pipeline of HOI detection, consisting of human-object pair detection and then
interaction classification. We define a set of errors and the oracles to fix
each of them. By measuring the mAP improvement obtained from fixing an error
using its oracle, we can have a detailed analysis of the significance of
different errors. We then delve into the human-object detection and interaction
classification, respectively, and check the model's behavior. For the first
detection task, we investigate both recall and precision, measuring the
coverage of ground-truth human-object pairs as well as the noisiness level in
the detections. For the second classification task, we compute mAP for
interaction classification only, without considering the detection scores. We
also measure the performance of the models in differentiating human-object
pairs with and without actual interactions using the AP (Average Precision)
score. Our toolbox is applicable for different methods across different
datasets and available at this https URL

我们介绍了一种用于分析现有 HOI 检测模型错误源的诊断工具箱，通过修复错误并测量 mAP 改进来详细分析不同错误的重要性。

诊断人 - 物体交互检测器

Diagnosing Human-object Interaction Detectors

Human-Object Interaction Detection tackles the problem of joint localization
and classification of human object interactions. Existing HOI transformers
either adopt a single decoder for triplet prediction, or utilize two parallel
decoders to detect individual objects and interactions separately, and compose
triplets by a matching process. In contrast, we decouple the triplet prediction
into human-object pair detection and interaction classification. Our main
motivation is that detecting the human-object instances and classifying
interactions accurately needs to learn representations that focus on different
regions. To this end, we present Disentangled Transformer, where both encoder
and decoder are disentangled to facilitate learning of two sub-tasks. To
associate the predictions of disentangled decoders, we first generate a unified
representation for HOI triplets with a base decoder, and then utilize it as
input feature of each disentangled decoder. Extensive experiments show that our
method outperforms prior work on two public HOI benchmarks by a sizeable
margin. Code will be available.

该研究旨在解决人 - 物交互检测中的联合定位和分类问题，通过提出解耦三元组预测为人 - 物对检测和交互分类两个子任务来学习关注不同区域的表示，从而设计出 Disentangled Transformer，与现有 HOI transformers 相比，其方法在两个公共 HOI 基准测试中表现更佳。