We investigate a new problem of detecting hands and recognizing their
physical contact state in unconstrained conditions. This is a challenging
inference task given the need to reason beyond the local appearance of hands.
The lack of training annotations indicating which object or parts of an object
the hand is in contact with further complicates the task. We propose a novel
convolutional network based on Mask-RCNN that can jointly learn to localize
hands and predict their physical contact to address this problem. The network
uses outputs from another object detector to obtain locations of objects
present in the scene. It uses these outputs and hand locations to recognize the
hand's contact state using two attention mechanisms. The first attention
mechanism is based on the hand and a region's affinity, enclosing the hand and
the object, and densely pools features from this region to the hand region. The
second attention module adaptively selects salient features from this plausible
region of contact. To develop and evaluate our method's performance, we
introduce a large-scale dataset called ContactHands, containing unconstrained
images annotated with hand locations and contact states. The proposed network,
including the parameters of attention modules, is end-to-end trainable. This
network achieves approximately 7\% relative improvement over a baseline network
that was built on the vanilla Mask-RCNN architecture and trained for
recognizing hand contact states.

本文提出一种基于卷积网络的手部检测与物体接触状态识别方法，采用两种注意力机制以及另一个物体检测器的输出，结合手部位置信息和场景中的物体位置信息，实现对手部接触状态的识别，并使用 ContactHands 数据集验证其优越性。