We present a novel deep learning-based approach to the 3D reconstruction of
clothed humans using weak supervision via 2D normal maps. Given a single RGB
image or multiview images, our network infers a signed distance function (SDF)
discretized on a tetrahedral mesh surrounding the body in a rest pose.
Subsequently, inferred pose and camera parameters are used to generate a normal
map from the SDF. A key aspect of our approach is the use of Marching
Tetrahedra to (uniquely) compute a triangulated surface from the SDF on the
tetrahedral mesh, facilitating straightforward differentiation (and thus
backpropagation). Thus, given only ground truth normal maps (with no volumetric
information ground truth information), we can train the network to produce SDF
values from corresponding RGB images. Optionally, an additional multiview loss
leads to improved results. We demonstrate the efficacy of our approach for both
network inference and 3D reconstruction.

使用 2D 法线贴图进行弱监督的基于深度学习的方法，实现服装人体的三维重建。利用神经网络推断在休息姿势下围绕身体离散的四面体网格上的有符号距离函数 (SDF)，接着利用推断得到的姿势和相机参数生成法线贴图。通过 Marching Tetrahedra 从四面体网格上的 SDF 计算出一个三角化表面，从而方便进行微分计算（反向传播）。因此，仅凭借真实的法线贴图（没有体积信息的真实数据），我们就可以训练网络从相应的 RGB 图像中生成 SDF 值。可选择性地使用多视角损失以改善结果。我们展示了该方法在网络推断和三维重建方面的有效性。

基于法线图的服装人体弱监督三维重建

Weakly-Supervised 3D Reconstruction of Clothed Humans via Normal Maps

There is a growing demand for the accessible creation of high-quality 3D
avatars that are animatable and customizable. Although 3D morphable models
provide intuitive control for editing and animation, and robustness for
single-view face reconstruction, they cannot easily capture geometric and
appearance details. Methods based on neural implicit representations, such as
signed distance functions (SDF) or neural radiance fields, approach
photo-realism, but are difficult to animate and do not generalize well to
unseen data. To tackle this problem, we propose a novel method for constructing
implicit 3D morphable face models that are both generalizable and intuitive for
editing. Trained from a collection of high-quality 3D scans, our face model is
parameterized by geometry, expression, and texture latent codes with a learned
SDF and explicit UV texture parameterization. Once trained, we can reconstruct
an avatar from a single in-the-wild image by leveraging the learned prior to
project the image into the latent space of our model. Our implicit morphable
face models can be used to render an avatar from novel views, animate facial
expressions by modifying expression codes, and edit textures by directly
painting on the learned UV-texture maps. We demonstrate quantitatively and
qualitatively that our method improves upon photo-realism, geometry, and
expression accuracy compared to state-of-the-art methods.

本文提出了一种构建隐式 3D 变形人脸模型的新方法，使用了学习得出的 SDF 和明确的 UV 贴图参数化，从而实现了单张图片的重建、面部表情动画的修改和纹理的直接绘制，相较于现有技术在照片逼真度、几何和表情精度上有了提升。