By equipping the most recent 3D Gaussian Splatting representation with head
3D morphable models (3DMM), existing methods manage to create head avatars with
high fidelity. However, most existing methods only reconstruct a head without
the body, substantially limiting their application scenarios. We found that
naively applying Gaussians to model the clothed chest and shoulders tends to
result in blurry reconstruction and noisy floaters under novel poses. This is
because of the fundamental limitation of Gaussians and point clouds -- each
Gaussian or point can only have a single directional radiance without spatial
variance, therefore an unnecessarily large number of them is required to
represent complicated spatially varying texture, even for simple geometry. In
contrast, we propose to model the body part with a neural texture that consists
of coarse and pose-dependent fine colors. To properly render the body texture
for each view and pose without accurate geometry nor UV mapping, we optimize
another sparse set of Gaussians as anchors that constrain the neural warping
field that maps image plane coordinates to the texture space. We demonstrate
that Gaussian Head & Shoulders can fit the high-frequency details on the
clothed upper body with high fidelity and potentially improve the accuracy and
fidelity of the head region. We evaluate our method with casual phone-captured
and internet videos and show our method archives superior reconstruction
quality and robustness in both self and cross reenactment tasks. To fully
utilize the efficient rendering speed of Gaussian splatting, we additionally
propose an accelerated inference method of our trained model without
Multi-Layer Perceptron (MLP) queries and reach a stable rendering speed of
around 130 FPS for any subjects.

通过在最新的 3D 高斯碎片表示上配备头部 3D 可塑模型（3DMM），现有方法成功地创建了高度逼真的头部化身。然而，大多数现有方法只能重建头部而不包括身体，从而严重限制了其应用场景。

高保真神经网络上半身头像：高斯锚定引导纹理扭曲

Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with  Anchor Gaussian Guided Texture Warping

We propose SelfRecon, a clothed human body reconstruction method that
combines implicit and explicit representations to recover space-time coherent
geometries from a monocular self-rotating human video. Explicit methods require
a predefined template mesh for a given sequence, while the template is hard to
acquire for a specific subject. Meanwhile, the fixed topology limits the
reconstruction accuracy and clothing types. Implicit representation supports
arbitrary topology and can represent high-fidelity geometry shapes due to its
continuous nature. However, it is difficult to integrate multi-frame
information to produce a consistent registration sequence for downstream
applications. We propose to combine the advantages of both representations. We
utilize differential mask loss of the explicit mesh to obtain the coherent
overall shape, while the details on the implicit surface are refined with the
differentiable neural rendering. Meanwhile, the explicit mesh is updated
periodically to adjust its topology changes, and a consistency loss is designed
to match both representations. Compared with existing methods, SelfRecon can
produce high-fidelity surfaces for arbitrary clothed humans with
self-supervised optimization. Extensive experimental results demonstrate its
effectiveness on real captured monocular videos. The source code is available
at this https URL

该研究提出了一种结合隐式和显式表达的身体重建方法 SelfRecon，可从单眼自旋人类视频中恢复空间 - 时间一致的几何形状，细节采用可微分神经渲染进行精细处理。

SelfRecon: 从单眼视频中自动重建数码头像

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

Statistical models of 3D human shape and pose learned from scan databases
have developed into valuable tools to solve a variety of vision and graphics
problems. Unfortunately, most publicly available models are of limited
expressiveness as they were learned on very small databases that hardly reflect
the true variety in human body shapes. In this paper, we contribute by
rebuilding a widely used statistical body representation from the largest
commercially available scan database, and making the resulting model available
to the community (visit this http URL). As preprocessing
several thousand scans for learning the model is a challenge in itself, we
contribute by developing robust best practice solutions for scan alignment that
quantitatively lead to the best learned models. We make implementations of
these preprocessing steps also publicly available. We extensively evaluate the
improved accuracy and generality of our new model, and show its improved
performance for human body reconstruction from sparse input data.

本文中，我们从最大可获得的商业扫描数据库中重建了一个广泛使用的统计人体表示，并使得产生的模型对社区可用。我们为了学习模型而预处理了几千次扫描，因此开发了强大的最佳实践解决方案，以排列扫描。我们在广泛的评估中展示出新模型的改进准确性和普遍性，并显示了它对于从稀疏输入数据中恢复人体的改进性能。