City scene generation has gained significant attention in autonomous driving,
smart city development, and traffic simulation. It helps enhance infrastructure
planning and monitoring solutions. Existing methods have employed a two-stage
process involving city layout generation, typically using Variational
Autoencoders (VAEs), Generative Adversarial Networks (GANs), or Transformers,
followed by neural rendering. These techniques often exhibit limited diversity
and noticeable artifacts in the rendered city scenes. The rendered scenes lack
variety, resembling the training images, resulting in monotonous styles.
Additionally, these methods lack planning capabilities, leading to less
realistic generated scenes. In this paper, we introduce CityCraft, an
innovative framework designed to enhance both the diversity and quality of
urban scene generation. Our approach integrates three key stages: initially, a
diffusion transformer (DiT) model is deployed to generate diverse and
controllable 2D city layouts. Subsequently, a Large Language Model(LLM) is
utilized to strategically make land-use plans within these layouts based on
user prompts and language guidelines. Based on the generated layout and city
plan, we utilize the asset retrieval module and Blender for precise asset
placement and scene construction. Furthermore, we contribute two new datasets
to the field: 1)CityCraft-OSM dataset including 2D semantic layouts of urban
areas, corresponding satellite images, and detailed annotations. 2)
CityCraft-Buildings dataset, featuring thousands of diverse, high-quality 3D
building assets. CityCraft achieves state-of-the-art performance in generating
realistic 3D cities.

CityCraft 是一种创新框架，结合了三个关键阶段来增强城市场景的多样性和质量，它包括使用扩散变压器模型生成 2D 城市布局，利用大型语言模型根据用户提示和语言指南制定土地利用规划，以及通过资产检索模块和 Blender 进行精确的资产放置和场景构建。CityCraft 在生成逼真的 3D 城市方面取得了最先进的性能。