We introduce SkelFormer, a novel markerless motion capture pipeline for multi-view human pose and shape estimation. Our method first uses off-the-shelf 2D keypoint estimators, pre-trained on large-scale in-the-wild data, to obtain 3D joint positions. Next, we design a regression-based inverse-kinematic skeletal transformer that maps the joint positions to pose and shape representations from heavily noisy observations. This module integrates prior knowledge about pose space and infers the full pose state at runtime. Separating the 3D keypoint detection and inverse-kinematic problems, along with the expressive representations learned by our skeletal transformer, enhance the generalization of our method to unseen noisy data. We evaluate our method on three public datasets in both in-distribution and out-of-distribution settings using three datasets, and observe strong performance with respect to prior works. Moreover, ablation experiments demonstrate the impact of each of the modules of our architecture. Finally, we study the performance of our method in dealing with noise and heavy occlusions and find considerable robustness with respect to other solutions.

为了多视角人体姿势和形状估计，我们介绍了SkelFormer，一种新颖的无标记动作捕捉方法。我们的方法首先使用现成的在野外大规模训练的2D关键点估计器来获取3D关节位置。接下来，我们设计了一个基于回归的逆运动学骨骼变换器，将关节位置从嘈杂的观测映射到姿势和形状表示。该模块结合了关于姿势空间的先验知识，并在运行时推断完整的姿势状态。通过将3D关键点检测和逆运动学问题分离，并利用我们骨骼变换器学到的表达能力，增强了我们的方法对未见过的嘈杂数据的泛化能力。我们在三个公共数据集上进行实验评估，在分布内和分布外的设置下，对之前的工作表现出良好的性能。此外，消融实验展示了我们架构的每个模块的影响。最后，我们研究了我们的方法在处理噪声和重叠方面的性能，并发现与其他解决方案相比具有相当的鲁棒性。

SkelFormer：使用骨架转换器进行无标记的3D姿态和形状估计