Despite the recent advancement of Generative Adversarial Networks (GANs) in
learning 3D-aware image synthesis from 2D data, existing methods fail to model
indoor scenes due to the large diversity of room layouts and the objects
inside. We argue that indoor scenes do not have a shared intrinsic structure,
and hence only using 2D images cannot adequately guide the model with the 3D
geometry. In this work, we fill in this gap by introducing depth as a 3D prior.
Compared with other 3D data formats, depth better fits the convolution-based
generation mechanism and is more easily accessible in practice. Specifically,
we propose a dual-path generator, where one path is responsible for depth
generation, whose intermediate features are injected into the other path as the
condition for appearance rendering. Such a design eases the 3D-aware synthesis
with explicit geometry information. Meanwhile, we introduce a switchable
discriminator both to differentiate real v.s. fake domains and to predict the
depth from a given input. In this way, the discriminator can take the spatial
arrangement into account and advise the generator to learn an appropriate depth
condition. Extensive experimental results suggest that our approach is capable
of synthesizing indoor scenes with impressively good quality and 3D
consistency, significantly outperforming state-of-the-art alternatives.

该研究提出了一种基于深度的双通道生成器和可切换鉴别器，用于从 2D 数据中合成 3D 感知室内场景的图像，并通过实验表明，该方法可以显著优于现有的最先进的方法。