Despite progress in human motion capture, existing multi-view methods often face challenges in estimating the 3D pose and shape of multiple closely interacting people. This difficulty arises from reliance on accurate 2D joint estimations, which are hard to obtain due to occlusions and body contact when people are in close interaction. To address this, we propose a novel method leveraging the personalized implicit neural avatar of each individual as a prior, which significantly improves the robustness and precision of this challenging pose estimation task. Concretely, the avatars are efficiently reconstructed via layered volume rendering from sparse multi-view videos. The reconstructed avatar prior allows for the direct optimization of 3D poses based on color and silhouette rendering loss, bypassing the issues associated with noisy 2D detections. To handle interpenetration, we propose a collision loss on the overlapping shape regions of avatars to add penetration constraints. Moreover, both 3D poses and avatars are optimized in an alternating manner. Our experimental results demonstrate state-of-the-art performance on several public datasets.

本研究解决了现有多视角方法在估计密切交互人群的三维姿态和形状时面临的困难，特别是由于遮挡和身体接触导致的2D关节点估计不准的问题。提出了一种新颖的方法，通过个体的隐式神经虚拟化身作为先验，显著提高了姿态估计的鲁棒性和精度。实验结果在多个公共数据集上展示了先进的性能。

AvatarPose：基于虚拟化身的稀疏多视角视频中密切人际交互的三维姿态估计