Human motion capture either requires multi-camera systems or is unreliable
using single-view input due to depth ambiguities. Meanwhile, mirrors are
readily available in urban environments and form an affordable alternative by
recording two views with only a single camera. However, the mirror setting
poses the additional challenge of handling occlusions of real and mirror image.
Going beyond existing mirror approaches for 3D human pose estimation, we
utilize mirrors for learning a complete body model, including shape and dense
appearance. Our main contributions are extending articulated neural radiance
fields to include a notion of a mirror, making it sample-efficient over
potential occlusion regions. Together, our contributions realize a
consumer-level 3D motion capture system that starts from off-the-shelf 2D poses
by automatically calibrating the camera, estimating mirror orientation, and
subsequently lifting 2D keypoint detections to 3D skeleton pose that is used to
condition the mirror-aware NeRF. We empirically demonstrate the benefit of
learning a body model and accounting for occlusion in challenging mirror
scenes.

利用镜子进行人体运动捕捉，包括通过学习一个完整的身体模型、自动校准摄像机、估计镜子方向和将 2D 关键点检测提升为 3D 骨骼姿势等，实现了一个消费级的 3D 运动捕捉系统，并在具有挑战性的镜子场景中实现了模型学习和遮挡的优势。