Existing 3D-from-2D generators are typically designed for well-curated
single-category datasets, where all the objects have (approximately) the same
scale, 3D location, and orientation, and the camera always points to the center
of the scene. This makes them inapplicable to diverse, in-the-wild datasets of
non-alignable scenes rendered from arbitrary camera poses. In this work, we
develop a 3D generator with Generic Priors (3DGP): a 3D synthesis framework
with more general assumptions about the training data, and show that it scales
to very challenging datasets, like ImageNet. Our model is based on three new
ideas. First, we incorporate an inaccurate off-the-shelf depth estimator into
3D GAN training via a special depth adaptation module to handle the
imprecision. Then, we create a flexible camera model and a regularization
strategy for it to learn its distribution parameters during training. Finally,
we extend the recent ideas of transferring knowledge from pre-trained
classifiers into GANs for patch-wise trained models by employing a simple
distillation-based technique on top of the discriminator. It achieves more
stable training than the existing methods and speeds up the convergence by at
least 40%. We explore our model on four datasets: SDIP Dogs 256x256, SDIP
Elephants 256x256, LSUN Horses 256x256, and ImageNet 256x256, and demonstrate
that 3DGP outperforms the recent state-of-the-art in terms of both texture and
geometry quality. Code and visualizations:
this https URL.

本文提出了一种基于 3D GAN 的 3D 生成器，采用了不同于以往的数据假设、深度框架和相机模型，在多样化数据集（如 ImageNet）上取得了优于现有技术的纹理和几何质量的结果。