We introduce a novel approach for 3D whole-body pose estimation, addressing the challenge of scale- and deformability- variance across body parts brought by the challenge of extending the 17 major joints on the human body to fine-grained keypoints on the face and hands. In addition to addressing the challenge of exploiting motion in unevenly sampled data, we combine stable diffusion to a hierarchical part representation which predicts the relative locations of fine-grained keypoints within each part (e.g., face) with respect to the part's local reference frame. On the H3WB dataset, our method greatly outperforms the current state of the art, which fails to exploit the temporal information. We also show considerable improvements compared to other spatiotemporal 3D human-pose estimation approaches that fail to account for the body part specificities. Code is available at https://github.com/valeoai/PAFUSE.

我们提出了一种新的方法来进行3D全身姿态估计，解决了由将人体的17个主要关节扩展到脸部和手部的细粒度关键点所带来的尺度和可变性差异的挑战。通过将稳定扩散与预测每个部分（例如，面部）内细粒度关键点相对于局部参考框架的相对位置的分层部分表示相结合，我们不仅解决了在不均匀采样数据中利用运动的挑战，还在H3WB数据集上大大超过了目前的艺术水平，并且与不考虑身体部分特异性的其他时空3D人体姿态估计方法相比，显示出了显著的改进。

PAFUSE: 基于部分传播的三维全身姿态估计