Reconstructing and tracking deformable surface with little or no texture has
posed long-standing challenges. Fundamentally, the challenges stem from
textureless surfaces lacking features for establishing cross-image
correspondences. In this work, we present a novel type of markers to
proactively enrich the object's surface features, and thereby ease the 3D
surface reconstruction and correspondence tracking. Our markers are made of
fluorescent dyes, visible only under the ultraviolet (UV) light and invisible
under regular lighting condition. Leveraging the markers, we design a
multi-camera system that captures surface deformation under the UV light and
the visible light in a time multiplexing fashion. Under the UV light, markers
on the object emerge to enrich its surface texture, allowing high-quality 3D
shape reconstruction and tracking. Under the visible light, markers become
invisible, allowing us to capture the object's original untouched appearance.
We perform experiments on various challenging scenes, including hand gestures,
facial expressions, waving cloth, and hand-object interaction. In all these
cases, we demonstrate that our system is able to produce robust, high-quality
3D reconstruction and tracking.

利用荧光染料制作的标记物为表面特征丰富提供了一种新方法，从而简化了三维表面重建和对应跟踪的过程，并展示了在不同场景下的稳健高质量的三维重建和跟踪能力。

无纹理可变形表面重建与隐形标记

Textureless Deformable Surface Reconstruction with Invisible Markers

We propose a method for predicting the 3D shape of a deformable surface from
a single view. By contrast with previous approaches, we do not need a
pre-registered template of the surface, and our method is robust to the lack of
texture and partial occlusions. At the core of our approach is a {\it
geometry-aware} deep architecture that tackles the problem as usually done in
analytic solutions: first perform 2D detection of the mesh and then estimate a
3D shape that is geometrically consistent with the image. We train this
architecture in an end-to-end manner using a large dataset of synthetic
renderings of shapes under different levels of deformation, material
properties, textures and lighting conditions. We evaluate our approach on a
test split of this dataset and available real benchmarks, consistently
improving state-of-the-art solutions with a significantly lower computational
time.

提出了一种基于几何感知的深度学习方法，能够在没有预先注册表面模板的情况下，通过单一视角的输入图像，精确预测可变形表面的 3D 形状， 并采用合成数据集和真实基准数据集的实验证明其在 3D 形状预测上超越同类算法，且计算时间明显减少。