Generating realistic 3D scenes is challenging due to the complexity of room
layouts and object geometries.We propose a sketch based knowledge enhanced
diffusion architecture (SEK) for generating customized, diverse, and plausible
3D scenes. SEK conditions the denoising process with a hand-drawn sketch of the
target scene and cues from an object relationship knowledge base. We first
construct an external knowledge base containing object relationships and then
leverage knowledge enhanced graph reasoning to assist our model in
understanding hand-drawn sketches. A scene is represented as a combination of
3D objects and their relationships, and then incrementally diffused to reach a
Gaussian distribution.We propose a 3D denoising scene transformer that learns
to reverse the diffusion process, conditioned by a hand-drawn sketch along with
knowledge cues, to regressively generate the scene including the 3D object
instances as well as their layout. Experiments on the 3D-FRONT dataset show
that our model improves FID, CKL by 17.41%, 37.18% in 3D scene generation and
FID, KID by 19.12%, 20.06% in 3D scene completion compared to the nearest
competitor DiffuScene.

基于手绘草图和对象关系知识的增强扩散架构 (SEK) 用于生成个性化、多样化和可信的 3D 场景，通过外部知识库和图推理辅助模型理解草图，利用 3D 去噪场景转换器生成包括对象实例和布局的场景。与竞争者相比，我们的模型在 3D 场景生成和完成中提高了 FID、CKL 17.41%、37.18% 以及 FID、KID 19.12%、20.06%。