Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics. However, due to occlusions and physical limitations of range sensors, existing methods only handle the visible surface, thus causing discontinuities and incompleteness in the motion field. To this end, we introduce 4DComplete, a novel data-driven approach that estimates the non-rigid motion for the unobserved geometry. 4DComplete takes as input a partial shape and motion observation, extracts 4D time-space embedding, and jointly infers the missing geometry and motion field using a sparse fully-convolutional network. For network training, we constructed a large-scale synthetic dataset called DeformingThings4D, which consists of 1972 animation sequences spanning 31 different animals or humanoid categories with dense 4D annotation. Experiments show that 4DComplete 1) reconstructs high-resolution volumetric shape and motion field from a partial observation, 2) learns an entangled 4D feature representation that benefits both shape and motion estimation, 3) yields more accurate and natural deformation than classic non-rigid priors such as As-Rigid-As-Possible (ARAP) deformation, and 4) generalizes well to unseen objects in real-world sequences.

使用倾斜感应器跟踪非刚性形变场景，介绍了一种新的数据驱动方法——4DComplete，它可以从局部形状和运动观测中，提取4D时空嵌入，并使用稀疏的全卷积网络共同推断缺失的几何和运动场，并通过大规模的综合数据集DeformingThings4D进行网络训练，实现从部分观测中重建高分辨率的体积形状和运动场、学习纠缠的4D特征表示以支持形状和运动估计、比经典非刚性先验deformation更准确自然，并且在真实场景中推广性好。

4DComplete: 超越可观测表面的非刚性运动估计