Human-object contact serves as a strong cue to understand how humans
physically interact with objects. Nevertheless, it is not widely explored to
utilize human-object contact information for the joint reconstruction of 3D
human and object from a single image. In this work, we present a novel joint 3D
human-object reconstruction method (CONTHO) that effectively exploits contact
information between humans and objects. There are two core designs in our
system: 1) 3D-guided contact estimation and 2) contact-based 3D human and
object refinement. First, for accurate human-object contact estimation, CONTHO
initially reconstructs 3D humans and objects and utilizes them as explicit 3D
guidance for contact estimation. Second, to refine the initial reconstructions
of 3D human and object, we propose a novel contact-based refinement Transformer
that effectively aggregates human features and object features based on the
estimated human-object contact. The proposed contact-based refinement prevents
the learning of erroneous correlation between human and object, which enables
accurate 3D reconstruction. As a result, our CONTHO achieves state-of-the-art
performance in both human-object contact estimation and joint reconstruction of
3D human and object. The code is publicly available at
this https URL

该研究提出了一种利用人 - 物接触信息进行三维人体与物体联合重建的新方法 (CONTHO)，通过 3D 引导接触估计和基于接触的三维人体与物体细化，实现了更准确的重建，取得了最先进的性能。

基于接触优化的转换器进行 3D 人体和物体联合重建

Joint Reconstruction of 3D Human and Object via Contact-Based Refinement  Transformer

Understanding how we grasp objects with our hands has important applications
in areas like robotics and mixed reality. However, this challenging problem
requires accurate modeling of the contact between hands and objects. To capture
grasps, existing methods use skeletons, meshes, or parametric models that can
cause misalignments resulting in inaccurate contacts. We present MANUS, a
method for Markerless Hand-Object Grasp Capture using Articulated 3D Gaussians.
We build a novel articulated 3D Gaussians representation that extends 3D
Gaussian splatting for high-fidelity representation of articulating hands.
Since our representation uses Gaussian primitives, it enables us to efficiently
and accurately estimate contacts between the hand and the object. For the most
accurate results, our method requires tens of camera views that current
datasets do not provide. We therefore build MANUS-Grasps, a new dataset that
contains hand-object grasps viewed from 53 cameras across 30+ scenes, 3
subjects, and comprising over 7M frames. In addition to extensive qualitative
results, we also show that our method outperforms others on a quantitative
contact evaluation method that uses paint transfer from the object to the hand.

使用关节化的 3D 高斯方法捕获手 - 物体抓取，建立 MANUS-Grasps 数据集并展示比其他方法更准确的联系评估方法。

MANUS：利用弯曲的 3D 高斯模型捕获无标记手 - 物体握取

MANUS: Markerless Hand-Object Grasp Capture using Articulated 3D  Gaussians

All hand-object interaction is controlled by forces that the two bodies exert
on each other, but little work has been done in modeling these underlying
forces when doing pose and contact estimation from RGB/RGB-D data. Given the
pose of the hand and object from any pose estimation system, we propose an
end-to-end differentiable model that refines pose estimates by learning the
forces experienced by the object at each vertex in its mesh. By matching the
learned net force to an estimate of net force based on finite differences of
position, this model is able to find forces that accurately describe the
movement of the object, while resolving issues like mesh interpenetration and
lack of contact. Evaluating on the ContactPose dataset, we show this model
successfully corrects poses and finds contact maps that better match the ground
truth, despite not using any RGB or depth image data.

本文提出了一种端到端可微模型，通过学习目标网格形状中每个顶点处的受力情况来修正手和物体的姿态，从而实现更准确的接触估计和姿态估计。