Unsupervised learning of 3D-aware generative adversarial networks (GANs)
using only collections of single-view 2D photographs has very recently made
much progress. These 3D GANs, however, have not been demonstrated for human
bodies and the generated radiance fields of existing frameworks are not
directly editable, limiting their applicability in downstream tasks. We propose
a solution to these challenges by developing a 3D GAN framework that learns to
generate radiance fields of human bodies or faces in a canonical pose and warp
them using an explicit deformation field into a desired body pose or facial
expression. Using our framework, we demonstrate the first high-quality radiance
field generation results for human bodies. Moreover, we show that our
deformation-aware training procedure significantly improves the quality of
generated bodies or faces when editing their poses or facial expressions
compared to a 3D GAN that is not trained with explicit deformations.

提出使用 3D GAN 框架对人体或脸部特征进行辐射场生成，通过显式变形场将其变形为期望的姿势或表情，并展示了编辑生成体或脸部特征时显式变形训练过程在提高其质量方面的显著优势。

生成式神经关节辐射场

Generative Neural Articulated Radiance Fields

We present a fully automatic system that takes a 3D scene and generates
plausible 3D human bodies that are posed naturally in that 3D scene. Given a 3D
scene without people, humans can easily imagine how people could interact with
the scene and the objects in it. However, this is a challenging task for a
computer as solving it requires that (1) the generated human bodies to be
semantically plausible within the 3D environment (e.g. people sitting on the
sofa or cooking near the stove), and (2) the generated human-scene interaction
to be physically feasible such that the human body and scene do not
interpenetrate while, at the same time, body-scene contact supports physical
interactions. To that end, we make use of the surface-based 3D human model
SMPL-X. We first train a conditional variational autoencoder to predict
semantically plausible 3D human poses conditioned on latent scene
representations, then we further refine the generated 3D bodies using scene
constraints to enforce feasible physical interaction. We show that our approach
is able to synthesize realistic and expressive 3D human bodies that naturally
interact with 3D environment. We perform extensive experiments demonstrating
that our generative framework compares favorably with existing methods, both
qualitatively and quantitatively. We believe that our scene-conditioned 3D
human generation pipeline will be useful for numerous applications; e.g. to
generate training data for human pose estimation, in video games and in VR/AR.
Our project page for data and code can be seen at:
https://vlg.inf.ethz.ch/projects/PSI/.

本研究提出了一种全自动系统，用于将 3D 场景转换成自然姿势的 3D 人体，该系统采用基于表面的 3D 人体模型 SMPL-X，利用条件变分自编码器和场景约束，可以合成与 3D 环境自然交互的真实、表现力强的 3D 人体，适用于许多应用领域，例如人体姿势估计、视频游戏和 VR / AR。