Inferring human-scene contact (HSC) is the first step toward understanding
how humans interact with their surroundings. While detecting 2D human-object
interaction (HOI) and reconstructing 3D human pose and shape (HPS) have enjoyed
significant progress, reasoning about 3D human-scene contact from a single
image is still challenging. Existing HSC detection methods consider only a few
types of predefined contact, often reduce body and scene to a small number of
primitives, and even overlook image evidence. To predict human-scene contact
from a single image, we address the limitations above from both data and
algorithmic perspectives. We capture a new dataset called RICH for "Real
scenes, Interaction, Contact and Humans." RICH contains multiview
outdoor/indoor video sequences at 4K resolution, ground-truth 3D human bodies
captured using markerless motion capture, 3D body scans, and high resolution 3D
scene scans. A key feature of RICH is that it also contains accurate
vertex-level contact labels on the body. Using RICH, we train a network that
predicts dense body-scene contacts from a single RGB image. Our key insight is
that regions in contact are always occluded so the network needs the ability to
explore the whole image for evidence. We use a transformer to learn such
non-local relationships and propose a new Body-Scene contact TRansfOrmer
(BSTRO). Very few methods explore 3D contact; those that do focus on the feet
only, detect foot contact as a post-processing step, or infer contact from body
pose without looking at the scene. To our knowledge, BSTRO is the first method
to directly estimate 3D body-scene contact from a single image. We demonstrate
that BSTRO significantly outperforms the prior art. The code and dataset are
available at this https URL

该研究通过构建一个新的名为 RICH 的数据集，使用 BSTRO 算法从单幅 RGB 图像中直接预测人体与场景的接触，在探索图像中覆盖的区域时，提出了一种新的非局部关系学习方法，并且通过实验证明了其比之前的方法具有更好的性能质。