Non-rigid structure-from-motion (NRSfM) has so far been mostly studied for
recovering 3D structure of a single non-rigid/deforming object. To handle the
real world challenging multiple deforming objects scenarios, existing methods
either pre-segment different objects in the scene or treat multiple non-rigid
objects as a whole to obtain the 3D non-rigid reconstruction. However, these
methods fail to exploit the inherent structure in the problem as the solution
of segmentation and the solution of reconstruction could not benefit each
other. In this paper, we propose a unified framework to jointly segment and
reconstruct multiple non-rigid objects. To compactly represent complex
multi-body non-rigid scenes, we propose to exploit the structure of the scenes
along both temporal direction and spatial direction, thus achieving a
spatio-temporal representation. Specifically, we represent the 3D non-rigid
deformations as lying in a union of subspaces along the temporal direction and
represent the 3D trajectories as lying in the union of subspaces along the
spatial direction. This spatio-temporal representation not only provides
competitive 3D reconstruction but also outputs robust segmentation of multiple
non-rigid objects. The resultant optimization problem is solved efficiently
using the Alternating Direction Method of Multipliers (ADMM). Extensive
experimental results on both synthetic and real multi-body NRSfM datasets
demonstrate the superior performance of our proposed framework compared with
the state-of-the-art methods.

本文提出了一种结合分割和重建多个非刚性物体的统一框架，采用时空表示来压缩复杂的多体非刚性场景，可以提供有竞争力的三维重建并输出多个非刚性物体的强鲁棒定位。