In this paper, we present Skeleton Transformer Networks (SkeletonNet), an
end-to-end framework that can predict not only 3D joint positions but also 3D
angular pose (bone rotations) of a human skeleton from a single color image.
This in turn allows us to generate skinned mesh animations. Here, we propose a
two-step regression approach. The first step regresses bone rotations in order
to obtain an initial solution by considering skeleton structure. The second
step performs refinement based on heatmap regressor using a 3D pose
representation called cross heatmap which stacks heatmaps of xy and zy
coordinates. By training the network using the proposed 3D human pose dataset
that is comprised of images annotated with 3D skeletal angular poses, we showed
that SkeletonNet can predict a full 3D human pose (joint positions and bone
rotations) from a single image in-the-wild.

该研究提出了一种名为 Skeleton Transformer Networks 的网络框架，可以通过单张彩色图像预测出人体骨架的 3D 关节点位置和 3D 角度姿态，并从而生成蒙皮网格动画。该框架采用两步回归方法进行骨骼旋转预测，其中第一步通过考虑骨架结构回归骨骼旋转以获得初始解，第二步则基于被称为交叉热图的三维姿态表示采用热图回归器进行精细化调整。实验表明，使用该框架训练的 3D 人体姿势数据集可以在野外环境下通过单张图像准确预测出人体的 3D 姿态。