We introduce DOPE, the first method to detect and estimate whole-body 3D
human poses, including bodies, hands and faces, in the wild. Achieving this
level of details is key for a number of applications that require understanding
the interactions of the people with each other or with the environment. The
main challenge is the lack of in-the-wild data with labeled whole-body 3D
poses. In previous work, training data has been annotated or generated for
simpler tasks focusing on bodies, hands or faces separately. In this work, we
propose to take advantage of these datasets to train independent experts for
each part, namely a body, a hand and a face expert, and distill their knowledge
into a single deep network designed for whole-body 2D-3D pose detection. In
practice, given a training image with partial or no annotation, each part
expert detects its subset of keypoints in 2D and 3D and the resulting
estimations are combined to obtain whole-body pseudo ground-truth poses. A
distillation loss encourages the whole-body predictions to mimic the experts'
outputs. Our results show that this approach significantly outperforms the same
whole-body model trained without distillation while staying close to the
performance of the experts. Importantly, DOPE is computationally less demanding
than the ensemble of experts and can achieve real-time performance. Test code
and models are available at
this https URL

DOPE 是第一种能够在复杂场景中检测和估计整体身体、手和面部 3D 姿势的方法，它利用独立的专家和蒸馏过程来训练整体姿势检测的单个深度神经网络，并在保持接近专家效果的同时实现了比大师集合更快的实时处理速度。