While AI-generated text and 2D images continue to expand its territory, 3D
generation has gradually emerged as a trend that cannot be ignored. Since the
year 2023 an abundant amount of research papers has emerged in the domain of 3D
generation. This growth encompasses not just the creation of 3D objects, but
also the rapid development of 3D character and motion generation. Several key
factors contribute to this progress. The enhanced fidelity in stable diffusion,
coupled with control methods that ensure multi-view consistency, and realistic
human models like SMPL-X, contribute synergistically to the production of 3D
models with remarkable consistency and near-realistic appearances. The
advancements in neural network-based 3D storing and rendering models, such as
Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have
accelerated the efficiency and realism of neural rendered models. Furthermore,
the multimodality capabilities of large language models have enabled language
inputs to transcend into human motion outputs. This paper aims to provide a
comprehensive overview and summary of the relevant papers published mostly
during the latter half year of 2023. It will begin by discussing the AI
generated object models in 3D, followed by the generated 3D human models, and
finally, the generated 3D human motions, culminating in a conclusive summary
and a vision for the future.

综述了 2023 年下半年发表的相关论文，主要包括 AI 生成的三维对象模型、三维人体模型和三维人体动作生成，并对未来进行了展望。

3D 生成智能的进展与前景：技术概述，包括 3D 人体

Progress and Prospects in 3D Generative AI: A Technical Overview  including 3D human

Generating 3D human models directly from text helps reduce the cost and time
of character modeling. However, achieving multi-attribute controllable and
realistic 3D human avatar generation is still challenging due to feature
coupling and the scarcity of realistic 3D human avatar datasets. To address
these issues, we propose Text2Avatar, which can generate realistic-style 3D
avatars based on the coupled text prompts. Text2Avatar leverages a discrete
codebook as an intermediate feature to establish a connection between text and
avatars, enabling the disentanglement of features. Furthermore, to alleviate
the scarcity of realistic style 3D human avatar data, we utilize a pre-trained
unconditional 3D human avatar generation model to obtain a large amount of 3D
avatar pseudo data, which allows Text2Avatar to achieve realistic style
generation. Experimental results demonstrate that our method can generate
realistic 3D avatars from coupled textual data, which is challenging for other
existing methods in this field.

使用 Text2Avatar 生成的 3D 真实风格人物化身，可通过耦合的文本数据生成逼真的 3D 角色模型，从而帮助减少人物建模的成本和时间。

Text2Avatar：基于编码本驱动的身体可控属性的文本生成三维人体化身

Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven  Body Controllable Attribute

We propose a new method for reconstructing controllable implicit 3D human
models from sparse multi-view RGB videos. Our method defines the neural scene
representation on the mesh surface points and signed distances from the surface
of a human body mesh. We identify an indistinguishability issue that arises
when a point in 3D space is mapped to its nearest surface point on a mesh for
learning surface-aligned neural scene representation. To address this issue, we
propose projecting a point onto a mesh surface using a barycentric
interpolation with modified vertex normals. Experiments with the ZJU-MoCap and
Human3.6M datasets show that our approach achieves a higher quality in a
novel-view and novel-pose synthesis than existing methods. We also demonstrate
that our method easily supports the control of body shape and clothes. Project
page: this https URL

本研究提出了一种从稀疏的多视角 RGB 视频中重建可控的隐式 3D 人体模型的新方法，使用网格表面点和身体网格的符号距离来定义神经场景表示，并通过改进的顶点法线的重心插值将点投影到网格表面上，实现了更高质量的新视角和新姿势合成，同时支持容易控制的身体形状和服装。

可控的 3D 人体合成的表面对齐神经辐射场

Surface-Aligned Neural Radiance Fields for Controllable 3D Human  Synthesis

We address the problem of fitting 3D human models to 3D scans of dressed
humans. Classical methods optimize both the data-to-model correspondences and
the human model parameters (pose and shape), but are reliable only when
initialized close to the solution. Some methods initialize the optimization
based on fully supervised correspondence predictors, which is not
differentiable end-to-end, and can only process a single scan at a time. Our
main contribution is LoopReg, an end-to-end learning framework to register a
corpus of scans to a common 3D human model. The key idea is to create a
self-supervised loop. A backward map, parameterized by a Neural Network,
predicts the correspondence from every scan point to the surface of the human
model. A forward map, parameterized by a human model, transforms the
corresponding points back to the scan based on the model parameters (pose and
shape), thus closing the loop. Formulating this closed loop is not
straightforward because it is not trivial to force the output of the NN to be
on the surface of the human model - outside this surface the human model is not
even defined. To this end, we propose two key innovations. First, we define the
canonical surface implicitly as the zero level set of a distance field in R3,
which in contrast to morecommon UV parameterizations, does not require cutting
the surface, does not have discontinuities, and does not induce distortion.
Second, we diffuse the human model to the 3D domain R3. This allows to map the
NN predictions forward,even when they slightly deviate from the zero level set.
Results demonstrate that we can train LoopRegmainly self-supervised - following
a supervised warm-start, the model becomes increasingly more accurate as
additional unlabelled raw scans are processed. Our code and pre-trained models
can be downloaded for research.

我们提出了 LoopReg，一种端到端的学习框架，用于将一组扫描数据对齐到一个共同的 3D 人体模型上。通过自学习的方式，我们成功地训练出了一个能够处理未标记原始扫描数据的模型。