Text-driven 3D indoor scene generation could be useful for gaming, film
industry, and AR/VR applications. However, existing methods cannot faithfully
capture the room layout, nor do they allow flexible editing of individual
objects in the room. To address these problems, we present Ctrl-Room, which is
able to generate convincing 3D rooms with designer-style layouts and
high-fidelity textures from just a text prompt. Moreover, Ctrl-Room enables
versatile interactive editing operations such as resizing or moving individual
furniture items. Our key insight is to separate the modeling of layouts and
appearance. %how to model the room that takes into account both scene texture
and geometry at the same time. To this end, Our proposed method consists of two
stages, a `Layout Generation Stage' and an `Appearance Generation Stage'. The
`Layout Generation Stage' trains a text-conditional diffusion model to learn
the layout distribution with our holistic scene code parameterization. Next,
the `Appearance Generation Stage' employs a fine-tuned ControlNet to produce a
vivid panoramic image of the room guided by the 3D scene layout and text
prompt. In this way, we achieve a high-quality 3D room with convincing layouts
and lively textures. Benefiting from the scene code parameterization, we can
easily edit the generated room model through our mask-guided editing module,
without expensive editing-specific training. Extensive experiments on the
Structured3D dataset demonstrate that our method outperforms existing methods
in producing more reasonable, view-consistent, and editable 3D rooms from
natural language prompts.

Ctrl-Room 通过使用文本提示生成具有设计风格布局和高保真纹理的令人信服的 3D 室内空间，并提供灵活的交互式编辑操作。通过分离布局和外观建模，并使用文本条件扩散模型和控制网络，我们能够轻松地编辑生成的房间模型，而不需要昂贵的编辑特定培训。在 Structured3D 数据集上的实验证明，我们的方法在从自然语言提示生成更合理、视觉一致和可编辑的 3D 房间方面优于现有方法。