We present the first single-network approach for 2D~whole-body pose estimation, which entails simultaneous localization of body, face, hands, and feet keypoints. Due to the bottom-up formulation, our method maintains constant real-time performance regardless of the number of people in the image. The network is trained in a single stage using multi-task learning, through an improved architecture which can handle scale differences between body/foot and face/hand keypoints. Our approach considerably improves upon OpenPose~\cite{cao2018openpose}, the only work so far capable of whole-body pose estimation, both in terms of speed and global accuracy. Unlike OpenPose, our method does not need to run an additional network for each hand and face candidate, making it substantially faster for multi-person scenarios. This work directly results in a reduction of computational complexity for applications that require 2D whole-body information (e.g., VR/AR, re-targeting). In addition, it yields higher accuracy, especially for occluded, blurry, and low resolution faces and hands. For code, trained models, and validation benchmarks, visit our project page: https://github.com/CMU-Perceptual-Computing-Lab/openpose_train.

本研究提出了一种新的单一神经网络方法，用于2D全身姿势估计，并可同时定位身体、面部、手和足关键点。该方法采用一种底部上的公式，使其在图像中的人数不论多少，都能保持恒定的实时性能。神经网络采用多任务学习，经过改进的结构可以处理身体/脚和面部/手关键点之间的尺度差异。这种方法在速度和全球精度方面都显著优于OpenPose，并且不像OpenPose那样需要为每个手和面候选者运行额外的网络，因此在多人场景中速度更快。本文的研究结果进一步降低了需要2D全身信息的应用程序（例如VR / AR，重新定位）的计算复杂性，而且在面部和手部模糊、低分辨率等情况下，准确率更高。

单网络全身姿态估计