Humans constantly contact objects to move and perform tasks. Thus, detecting
human-object contact is important for building human-centered artificial
intelligence. However, there exists no robust method to detect contact between
the body and the scene from an image, and there exists no dataset to learn such
a detector. We fill this gap with HOT ("Human-Object conTact"), a new dataset
of human-object contacts for images. To build HOT, we use two data sources: (1)
We use the PROX dataset of 3D human meshes moving in 3D scenes, and
automatically annotate 2D image areas for contact via 3D mesh proximity and
projection. (2) We use the V-COCO, HAKE and Watch-n-Patch datasets, and ask
trained annotators to draw polygons for the 2D image areas where contact takes
place. We also annotate the involved body part of the human body. We use our
HOT dataset to train a new contact detector, which takes a single color image
as input, and outputs 2D contact heatmaps as well as the body-part labels that
are in contact. This is a new and challenging task that extends current
foot-ground or hand-object contact detectors to the full generality of the
whole body. The detector uses a part-attention branch to guide contact
estimation through the context of the surrounding body parts and scene. We
evaluate our detector extensively, and quantitative results show that our model
outperforms baselines, and that all components contribute to better
performance. Results on images from an online repository show reasonable
detections and generalizability.

本文介绍了一种用于检测人与周围物体接触的新数据集 HOT，并使用该数据集训练了一个新的接触检测器，它可以输出 2D 接触热图以及处于接触状态的身体部位标签。该检测器采用了部分注意分支来通过周围身体部分和场景的上下文指导接触估计，并且定量的结果显示，该模型的性能优于基线，并且所有组件都对提高性能有贡献。