We propose L3DG, the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. This enables effective generative 3D modeling, scaling to generation of entire room-scale scenes which can be very efficiently rendered. To enable effective synthesis of 3D Gaussians, we propose a latent diffusion formulation, operating in a compressed latent space of 3D Gaussians. This compressed latent space is learned by a vector-quantized variational autoencoder (VQ-VAE), for which we employ a sparse convolutional architecture to efficiently operate on room-scale scenes. This way, the complexity of the costly generation process via diffusion is substantially reduced, allowing higher detail on object-level generation, as well as scalability to large scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time. We demonstrate that our approach significantly improves visual quality over prior work on unconditional object-level radiance field synthesis and showcase its applicability to room-scale scene generation.

本文提出了L3DG，这是第一种通过潜在三维高斯扩散形式进行三维高斯生成建模的方法，填补了现有技术在生成室内场景方面的空白。该方法采用压缩的潜在空间，通过稀疏卷积架构实现高效的三维高斯合成，成功降低了生成过程的复杂性，显著提升了视觉质量并支持实时渲染，对室内场景生成具有重要影响。

L3DG：潜在三维高斯扩散