Camera captured human pose is an outcome of several sources of variation.
Performance of supervised 3D pose estimation approaches comes at the cost of
dispensing with variations, such as shape and appearance, that may be useful
for solving other related tasks. As a result, the learned model not only
inculcates task-bias but also dataset-bias because of its strong reliance on
the annotated samples, which also holds true for weakly-supervised models.
Acknowledging this, we propose a self-supervised learning framework to
disentangle such variations from unlabeled video frames. We leverage the prior
knowledge on human skeleton and poses in the form of a single part-based 2D
puppet model, human pose articulation constraints, and a set of unpaired 3D
poses. Our differentiable formalization, bridging the representation gap
between the 3D pose and spatial part maps, not only facilitates discovery of
interpretable pose disentanglement but also allows us to operate on videos with
diverse camera movements. Qualitative results on unseen in-the-wild datasets
establish our superior generalization across multiple tasks beyond the primary
tasks of 3D pose estimation and part segmentation. Furthermore, we demonstrate
state-of-the-art weakly-supervised 3D pose estimation performance on both
Human3.6M and MPI-INF-3DHP datasets.

本文提出了一种基于自监督学习的框架，利用单个基于零件的 2D 木偶模型、人体姿势约束和一组未配对的 3D 姿势来区分摄像机捕捉的人体姿态中的变化，从而在不同视频中提高多个任务的性能，包括 3D 姿势估计和部分分割。

基于部件导向的新颖图像合成的自监督三维人体姿态估计

Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image  Synthesis

In this paper, we present Skeleton Transformer Networks (SkeletonNet), an
end-to-end framework that can predict not only 3D joint positions but also 3D
angular pose (bone rotations) of a human skeleton from a single color image.
This in turn allows us to generate skinned mesh animations. Here, we propose a
two-step regression approach. The first step regresses bone rotations in order
to obtain an initial solution by considering skeleton structure. The second
step performs refinement based on heatmap regressor using a 3D pose
representation called cross heatmap which stacks heatmaps of xy and zy
coordinates. By training the network using the proposed 3D human pose dataset
that is comprised of images annotated with 3D skeletal angular poses, we showed
that SkeletonNet can predict a full 3D human pose (joint positions and bone
rotations) from a single image in-the-wild.

该研究提出了一种名为 Skeleton Transformer Networks 的网络框架，可以通过单张彩色图像预测出人体骨架的 3D 关节点位置和 3D 角度姿态，并从而生成蒙皮网格动画。该框架采用两步回归方法进行骨骼旋转预测，其中第一步通过考虑骨架结构回归骨骼旋转以获得初始解，第二步则基于被称为交叉热图的三维姿态表示采用热图回归器进行精细化调整。实验表明，使用该框架训练的 3D 人体姿势数据集可以在野外环境下通过单张图像准确预测出人体的 3D 姿态。