Learning general image representations has proven key to the success of many
computer vision tasks. For example, many approaches to image understanding
problems rely on deep networks that were initially trained on ImageNet, mostly
because the learned features are a valuable starting point to learn from
limited labeled data. However, when it comes to 3D motion capture of multiple
people, these features are only of limited use.
In this paper, we therefore propose an approach to learning features that are
useful for this purpose. To this end, we introduce a self-supervised approach
to learning what we call a neural scene decomposition (NSD) that can be
exploited for 3D pose estimation. NSD comprises three layers of abstraction to
represent human subjects: spatial layout in terms of bounding-boxes and
relative depth; a 2D shape representation in terms of an instance segmentation
mask; and subject-specific appearance and 3D pose information. By exploiting
self-supervision coming from multiview data, our NSD model can be trained
end-to-end without any 2D or 3D supervision. In contrast to previous
approaches, it works for multiple persons and full-frame images. Because it
encodes 3D geometry, NSD can then be effectively leveraged to train a 3D pose
estimation network from small amounts of annotated data.

本文提出一种基于自监督学习的神经场景分解（NSD）方法，通过对多视角数据的学习，用三层信息表示人体姿态，其中包括边界框和相对深度，实例分割掩码和个体化外观信息以及 3D 姿态信息，该方法能够为小规模数据集训练 3D 姿态估计网络提供帮助。