Recent advances in generative diffusion models have enabled the previously
unfeasible capability of generating 3D assets from a single input image or a
text prompt. In this work, we aim to enhance the quality and functionality of
these models for the task of creating controllable, photorealistic human
avatars. We achieve this by integrating a 3D morphable model into the
state-of-the-art multiview-consistent diffusion approach. We demonstrate that
accurate conditioning of a generative pipeline on the articulated 3D model
enhances the baseline model performance on the task of novel view synthesis
from a single image. More importantly, this integration facilitates a seamless
and accurate incorporation of facial expression and body pose control into the
generation process. To the best of our knowledge, our proposed framework is the
first diffusion model to enable the creation of fully 3D-consistent,
animatable, and photorealistic human avatars from a single image of an unseen
subject; extensive quantitative and qualitative evaluations demonstrate the
advantages of our approach over existing state-of-the-art avatar creation
models on both novel view and novel expression synthesis tasks.

本研究工作旨在通过将 3D 可变模型整合到最新的多视角一致性扩散方法中，增强生成扩散模型在创建可控、照片般逼真的人类头像任务中的质量和功能。我们的实验证明了在基于关节的 3D 模型的准确约束下，生成流水线模型在单图像的新视角合成任务上的性能改进，更重要的是，这种整合实现了面部表情和身体姿势控制在生成过程中的无缝和准确融入。据我们所知，我们提出的框架是第一个允许从未见过的单一图像创建完全 3D 一致、可动画和照片般逼真的人类头像的扩散模型；广泛的定量和定性评估证明了我们的方法在新视角和新表情合成任务上相对于现有最先进的头像创建模型的优势。

可塑扩散：单图像化身创造的三维一致扩散

Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar  Creation

Real-time rendering of photorealistic and controllable human avatars stands
as a cornerstone in Computer Vision and Graphics. While recent advances in
neural implicit rendering have unlocked unprecedented photorealism for digital
avatars, real-time performance has mostly been demonstrated for static scenes
only. To address this, we propose ASH, an animatable Gaussian splatting
approach for photorealistic rendering of dynamic humans in real-time. We
parameterize the clothed human as animatable 3D Gaussians, which can be
efficiently splatted into image space to generate the final rendering. However,
naively learning the Gaussian parameters in 3D space poses a severe challenge
in terms of compute. Instead, we attach the Gaussians onto a deformable
character model, and learn their parameters in 2D texture space, which allows
leveraging efficient 2D convolutional architectures that easily scale with the
required number of Gaussians. We benchmark ASH with competing methods on
pose-controllable avatars, demonstrating that our method outperforms existing
real-time methods by a large margin and shows comparable or even better results
than offline methods.

提议了 ASH 方法，一种用于实时渲染动态人物的可控逼真的动画高斯样本化方法，相比其他现有的实时方法，在姿势可控的人物形象方面表现出色，与离线方法相比表现相当甚至更好。