Understanding how humans use physical contact to interact with the world is
key to enabling human-centric artificial intelligence. While inferring 3D
contact is crucial for modeling realistic and physically-plausible human-object
interactions, existing methods either focus on 2D, consider body joints rather
than the surface, use coarse 3D body regions, or do not generalize to
in-the-wild images. In contrast, we focus on inferring dense, 3D contact
between the full body surface and objects in arbitrary images. To achieve this,
we first collect DAMON, a new dataset containing dense vertex-level contact
annotations paired with RGB images containing complex human-object and
human-scene contact. Second, we train DECO, a novel 3D contact detector that
uses both body-part-driven and scene-context-driven attention to estimate
vertex-level contact on the SMPL body. DECO builds on the insight that human
observers recognize contact by reasoning about the contacting body parts, their
proximity to scene objects, and the surrounding scene context. We perform
extensive evaluations of our detector on DAMON as well as on the RICH and
BEHAVE datasets. We significantly outperform existing SOTA methods across all
benchmarks. We also show qualitatively that DECO generalizes well to diverse
and challenging real-world human interactions in natural images. The code,
data, and models are available at this https URL

理解人类如何通过身体接触与世界互动对于实现以人为中心的人工智能至关重要。我们关注于推断全身表面与任意图像中的物体之间的密集 3D 接触，通过构建新的数据集 DAMON 以及训练新的 3D 接触检测器 DECO，在各种基准测试中显著优于现有方法，同时展示 DECO 在自然图像中广泛适用于多样且具有挑战性的真实人类互动。

DECO: 野外环境下 3D 人体场景接触密集估计

DECO: Dense Estimation of 3D Human-Scene Contact In The Wild

Understanding 3d human interactions is fundamental for fine-grained scene
analysis and behavioural modeling. However, most of the existing models predict
incorrect, lifeless 3d estimates, that miss the subtle human contact
aspects--the essence of the event--and are of little use for detailed
behavioral understanding. This paper addresses such issues with several
contributions: (1) we introduce models for interaction signature estimation
(ISP) encompassing contact detection, segmentation, and 3d contact signature
prediction; (2) we show how such components can be leveraged to ensure contact
consistency during 3d reconstruction; (3) we construct several large datasets
for learning and evaluating 3d contact prediction and reconstruction methods;
specifically, we introduce CHI3D, a lab-based accurate 3d motion capture
dataset with 631 sequences containing $2,525$ contact events, $728,664$ ground
truth 3d poses, as well as FlickrCI3D, a dataset of $11,216$ images, with
$14,081$ processed pairs of people, and $81,233$ facet-level surface
correspondences. Finally, (4) we propose methodology for recovering the
ground-truth pose and shape of interacting people in a controlled setup and (5)
annotate all 3d interaction motions in CHI3D with textual descriptions. Motion
data in multiple formats (GHUM and SMPLX parameters, Human3.6m 3d joints) is
made available for research purposes at https://ci3d.imar.ro, together
with an evaluation server and a public benchmark.

本文研究了 3D 人类交互的意义和行为建模，并通过引入模型和数据集等多个方面的贡献，提出了一种解决 3D 重建中遗漏人体细微接触等问题的方法。

重建相互作用人体的三维模型

Reconstructing Three-Dimensional Models of Interacting Humans

In this article, we propose a deep learning framework that provides a unified
approach to the problem of leg contact detection in humanoid robot walking
gaits. Our formulation accomplishes to accurately and robustly estimate the
contact state probability for each leg (i.e., stable or slip/no contact). The
proposed framework employs solely proprioceptive sensing and although it relies
on simulated ground-truth contact data for the classification process, we
demonstrate that it generalizes across varying friction surfaces and different
legged robotic platforms and, at the same time, is readily transferred from
simulation to practice. The framework is quantitatively and qualitatively
assessed in simulation via the use of ground-truth contact data and is
contrasted against state of-the-art methods with an ATLAS, a NAO, and a TALOS
humanoid robot. Furthermore, its efficacy is demonstrated in base estimation
with a real TALOS humanoid. To reinforce further research endeavors, our
implementation is offered as an open-source ROS/Python package, coined Legged
Contact Detection (LCD).

本文提出了一个深度学习框架来解决人形机器人步态中的腿部接触检测问题。该框架利用了本体感知，能够准确和稳健地估计每条腿的接触状态概率，并通过模拟得到的接触数据进行分类处理。该框架在模拟环境中通过对地面真实接触数据的使用进行了定量和定性评估，并对比了目前的先进方法。同时，它被证明在基本估计方面效果显著。