Marker-based and marker-less optical skeletal motion-capture methods use an
outside-in arrangement of cameras placed around a scene, with viewpoints
converging on the center. They often create discomfort by possibly needed
marker suits, and their recording volume is severely restricted and often
constrained to indoor scenes with controlled backgrounds. We therefore propose
a new method for real-time, marker-less and egocentric motion capture which
estimates the full-body skeleton pose from a lightweight stereo pair of fisheye
cameras that are attached to a helmet or virtual-reality headset. It combines
the strength of a new generative pose estimation framework for fisheye views
with a ConvNet-based body-part detector trained on a new automatically
annotated and augmented dataset. Our inside-in method captures full-body motion
in general indoor and outdoor scenes, and also crowded scenes.

提出一种新的方法，可以在室内和室外等各种场景下，实时地通过佩戴在头盔或虚拟现实头戴式显示器上的轻量级鱼眼立体相机，使用生成式姿势估计和基于 ConvNet 的身体部位检测器，无需标记，从而捕捉全身运动。

EgoCap: 两个鱼眼相机实现无标记主观动作捕捉（扩展摘要）

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras  (Extended Abstract)

We propose an heterogeneous multi-task learning framework for human pose
estimation from monocular image with deep convolutional neural network. In
particular, we simultaneously learn a pose-joint regressor and a sliding-window
body-part detector in a deep network architecture. We show that including the
body-part detection task helps to regularize the network, directing it to
converge to a good solution. We report competitive and state-of-art results on
several data sets. We also empirically show that the learned neurons in the
middle layer of our network are tuned to localized body parts.

本文提出了一个异构多任务学习框架，用于通过深度卷积神经网络从单 ocular 图像中实现人体姿势估计，同时学习姿势 - 关节回归器和滑动窗口身体部位检测器，并证明包含身体部位检测任务有助于正则化网络，并指导其收敛于良好解决方案，并在多个数据集上报告了竞争和最先进的结果。 还通过实验证明，我们的网络中间层中学习的神经元针对局部身体部位进行了优化。