3D reconstruction of dynamic scenes is a long-standing problem in computer
graphics and increasingly difficult the less information is available.
Shape-from-Template (SfT) methods aim to reconstruct a template-based geometry
from RGB images or video sequences, often leveraging just a single monocular
camera without depth information, such as regular smartphone recordings.
Unfortunately, existing reconstruction methods are either unphysical and noisy
or slow in optimization. To solve this problem, we propose a novel SfT
reconstruction algorithm for cloth using a pre-trained neural surrogate model
that is fast to evaluate, stable, and produces smooth reconstructions due to a
regularizing physics simulation. Differentiable rendering of the simulated mesh
enables pixel-wise comparisons between the reconstruction and a target video
sequence that can be used for a gradient-based optimization procedure to
extract not only shape information but also physical parameters such as
stretching, shearing, or bending stiffness of the cloth. This allows to retain
a precise, stable, and smooth reconstructed geometry while reducing the runtime
by a factor of 400-500 compared to $\phi$-SfT, a state-of-the-art physics-based
SfT approach.

基于预训练神经替代模型的快速和稳定的织物三维重建算法，可精确、稳定、平滑地重建几何体，与物理基础的 SfT 方法相比，运行时间减少了 400-500 倍。

基于物理引导的模板形状恢复：通过神经替代模型的单目视频感知

Physics-guided Shape-from-Template: Monocular Video Perception through  Neural Surrogate Models

We propose united implicit functions (UNIF), a part-based method for clothed
human reconstruction and animation with raw scans and skeletons as the input.
Previous part-based methods for human reconstruction rely on ground-truth part
labels from SMPL and thus are limited to minimal-clothed humans. In contrast,
our method learns to separate parts from body motions instead of part
supervision, thus can be extended to clothed humans and other articulated
objects. Our Partition-from-Motion is achieved by a bone-centered
initialization, a bone limit loss, and a section normal loss that ensure stable
part division even when the training poses are limited. We also present a
minimal perimeter loss for SDF to suppress extra surfaces and part overlapping.
Another core of our method is an adjacent part seaming algorithm that produces
non-rigid deformations to maintain the connection between parts which
significantly relieves the part-based artifacts. Under this algorithm, we
further propose "Competing Parts", a method that defines blending weights by
the relative position of a point to bones instead of the absolute position,
avoiding the generalization problem of neural implicit functions with inverse
LBS (linear blend skinning). We demonstrate the effectiveness of our method by
clothed human body reconstruction and animation on the CAPE and the ClothSeq
datasets.

该研究提出一种适用于带衣物人体建模与动画的联合隐式函数 (part-based method)，包括骨架初始化和稳定分区方式以及非刚性变形算法，采用相对位置权重来定义混合权值以防止神经隐式函数中的反向 LBS 常规化问题。相关实验表明，该方法对 CAPE 和 ClothSeq 数据集的还原和动画具有显著的效果。