Controllable generation of 3D assets is important for many practical
applications like content creation in movies, games and engineering, as well as
in AR/VR. Recently, diffusion models have shown remarkable results in
generation quality of 3D objects. However, none of the existing models enable
disentangled generation to control the shape and appearance separately. For the
first time, we present a suitable representation for 3D diffusion models to
enable such disentanglement by introducing a hybrid point cloud and neural
radiance field approach. We model a diffusion process over point positions
jointly with a high-dimensional feature space for a local density and radiance
decoder. While the point positions represent the coarse shape of the object,
the point features allow modeling the geometry and appearance details. This
disentanglement enables us to sample both independently and therefore to
control both separately. Our approach sets a new state of the art in generation
compared to previous disentanglement-capable methods by reduced FID scores of
30-90% and is on-par with other non disentanglement-capable state-of-the art
methods.

在电影、游戏、工程以及增强 / 虚拟现实等多种实际应用中，可控地生成 3D 资产具有重要意义。最近，扩散模型在生成 3D 对象的质量方面取得了显著的成果。然而，现有模型中没有一个能够实现对形状和外观的分离生成。我们首次提出了一种适用于 3D 扩散模型的合适表示方法，通过引入混合的点云和神经辐射场方法实现了这种分离，模拟了点位置的扩散过程，并结合高维特征空间的局部密度和亮度解码器。点位置表示对象的粗略形状，而点特征则允许对几何形状和外观细节进行建模。这种分离使得我们能够独立地对二者进行采样和控制。与之前的分离能力方法相比，我们的方法在生成方面达到了新的技术水平，降低了 30-90% 的 FID 分数，并且与其他非分离能力的最先进方法处于同一水平。