We propose a novel generative adversarial network (GAN) for the task of
unsupervised learning of 3D representations from natural images. Most
generative models rely on 2D kernels to generate images and make few
assumptions about the 3D world. These models therefore tend to create blurry
images or artefacts in tasks that require a strong 3D understanding, such as
novel-view synthesis. HoloGAN instead learns a 3D representation of the world,
and to render this representation in a realistic manner. Unlike other GANs,
HoloGAN provides explicit control over the pose of generated objects through
rigid-body transformations of the learnt 3D features. Our experiments show that
using explicit 3D features enables HoloGAN to disentangle 3D pose and identity,
which is further decomposed into shape and appearance, while still being able
to generate images with similar or higher visual quality than other generative
models. HoloGAN can be trained end-to-end from unlabelled 2D images only.
Particularly, we do not require pose labels, 3D shapes, or multiple views of
the same objects. This shows that HoloGAN is the first generative model that
learns 3D representations from natural images in an entirely unsupervised
manner.

本文提出了一种新颖 的生成对抗网络 HoloGAN，它使用 3D 表示学习世界，而不使用 2D 核心进行图像合成，可以对生成的物体姿态进行精确控制。与其他 GAN 不同，HoloGAN 可以在不需要标注姿态标签、3D 形状或同一对象的多个视角之前，仅从未标记的 2D 图像中端到端地训练。经实验证明，HoloGAN 可以在完全无监督的情况下从自然图像中学习 3D 表示，具有较高的视觉质量。