Estimating hand-object manipulations is essential for interpreting and
imitating human actions. Previous work has made significant progress towards
reconstruction of hand poses and object shapes in isolation. Yet,
reconstructing hands and objects during manipulation is a more challenging task
due to significant occlusions of both the hand and object. While presenting
challenges, manipulations may also simplify the problem since the physics of
contact restricts the space of valid hand-object configurations. For example,
during manipulation, the hand and object should be in contact but not
interpenetrate. In this work, we regularize the joint reconstruction of hands
and objects with manipulation constraints. We present an end-to-end learnable
model that exploits a novel contact loss that favors physically plausible
hand-object constellations. Our approach improves grasp quality metrics over
baselines, using RGB images as input. To train and evaluate the model, we also
propose a new large-scale synthetic dataset, ObMan, with hand-object
manipulations. We demonstrate the transferability of ObMan-trained models to
real data.

基于物理约束的手 - 物体重构模型，利用合成数据集 ObMan 进行模型训练和验证，达到手 - 物交互场景下的稳定手部姿势和可行物体形状重构。