Recent techniques on implicit geometry representation learning and neural
rendering have shown promising results for 3D clothed human reconstruction from
sparse video inputs. However, it is still challenging to reconstruct detailed
surface geometry and even more difficult to synthesize photorealistic novel
views with animated human poses. In this work, we introduce PGAHum, a
prior-guided geometry and appearance learning framework for high-fidelity
animatable human reconstruction. We thoroughly exploit 3D human priors in three
key modules of PGAHum to achieve high-quality geometry reconstruction with
intricate details and photorealistic view synthesis on unseen poses. First, a
prior-based implicit geometry representation of 3D human, which contains a
delta SDF predicted by a tri-plane network and a base SDF derived from the
prior SMPL model, is proposed to model the surface details and the body shape
in a disentangled manner. Second, we introduce a novel prior-guided sampling
strategy that fully leverages the prior information of the human pose and body
to sample the query points within or near the body surface. By avoiding
unnecessary learning in the empty 3D space, the neural rendering can recover
more appearance details. Last, we propose a novel iterative backward
deformation strategy to progressively find the correspondence for the query
point in observation space. A skinning weights prediction model is learned
based on the prior provided by the SMPL model to achieve the iterative backward
LBS deformation. Extensive quantitative and qualitative comparisons on various
datasets are conducted and the results demonstrate the superiority of our
framework. Ablation studies also verify the effectiveness of each scheme for
geometry and appearance learning.

我们介绍了 PGAHum，一种先验引导的几何与外观学习框架，用于高保真度可动人体重建。我们在三个关键模块中充分利用 3D 人体先验，实现了具有细节丰富的几何重建和能在未见过的姿势上合成逼真视图的高质量结果。

PGAHum：先验引导的几何与外观学习用于高保真可动人物重建

PGAHum: Prior-Guided Geometry and Appearance Learning for High-Fidelity  Animatable Human Reconstruction

Realistic virtual humans play a crucial role in numerous industries, such as
metaverse, intelligent healthcare, and self-driving simulation. But creating
them on a large scale with high levels of realism remains a challenge. The
utilization of deep implicit function sparks a new era of image-based 3D
clothed human reconstruction, enabling pixel-aligned shape recovery with fine
details. Subsequently, the vast majority of works locate the surface by
regressing the deterministic implicit value for each point. However, should all
points be treated equally regardless of their proximity to the surface? In this
paper, we propose replacing the implicit value with an adaptive uncertainty
distribution, to differentiate between points based on their distance to the
surface. This simple ``value to distribution'' transition yields significant
improvements on nearly all the baselines. Furthermore, qualitative results
demonstrate that the models trained using our uncertainty distribution loss,
can capture more intricate wrinkles, and realistic limbs. Code and models are
available for research purposes at this https URL

大规模高度真实感的 3D 服装人重建基于深度隐函数，利用自适应的不确定性分布提高模型的内在真实感和 纹理细节。

D-IF: 面向不确定性感知的人体数字化通过隐式分布场

D-IF: Uncertainty-aware Human Digitization via Implicit Distribution  Field

Although much progress has been made in 3D clothed human reconstruction, most
of the existing methods fail to produce robust results from in-the-wild images,
which contain diverse human poses and appearances. This is mainly due to the
large domain gap between training datasets and in-the-wild datasets. The
training datasets are usually synthetic ones, which contain rendered images
from GT 3D scans. However, such datasets contain simple human poses and less
natural image appearances compared to those of real in-the-wild datasets, which
makes generalization of it to in-the-wild images extremely challenging. To
resolve this issue, in this work, we propose ClothWild, a 3D clothed human
reconstruction framework that firstly addresses the robustness on in-thewild
images. First, for the robustness to the domain gap, we propose a weakly
supervised pipeline that is trainable with 2D supervision targets of
in-the-wild datasets. Second, we design a DensePose-based loss function to
reduce ambiguities of the weak supervision. Extensive empirical tests on
several public in-the-wild datasets demonstrate that our proposed ClothWild
produces much more accurate and robust results than the state-of-the-art
methods. The codes are available in here:
this https URL

本论文提出了一种基于 ClothWild 的 3D 服装人体重建框架，通过一种弱监督管道和 DensePose-based 损失函数解决了 training 和 in-the-wild 数据之间的大型领域差距，在公开数据集上得到了比现有技术更准确和更健壮的结果。