Understanding the evolution of 3D scenes is important for effective
autonomous driving. While conventional methods mode scene development with the
motion of individual instances, world models emerge as a generative framework
to describe the general scene dynamics. However, most existing methods adopt an
autoregressive framework to perform next-token prediction, which suffer from
inefficiency in modeling long-term temporal evolutions. To address this, we
propose a diffusion-based 4D occupancy generation model, OccSora, to simulate
the development of the 3D world for autonomous driving. We employ a 4D scene
tokenizer to obtain compact discrete spatial-temporal representations for 4D
occupancy input and achieve high-quality reconstruction for long-sequence
occupancy videos. We then learn a diffusion transformer on the spatial-temporal
representations and generate 4D occupancy conditioned on a trajectory prompt.
We conduct extensive experiments on the widely used nuScenes dataset with Occ3D
occupancy annotations. OccSora can generate 16s-videos with authentic 3D layout
and temporal consistency, demonstrating its ability to understand the spatial
and temporal distributions of driving scenes. With trajectory-aware 4D
generation, OccSora has the potential to serve as a world simulator for the
decision-making of autonomous driving. Code is available at:
this https URL

理解 3D 场景的演变对于有效的自动驾驶至关重要。我们提出了一种基于扩散的 4D 占据生成模型 OccSora，以模拟自动驾驶中三维世界的发展。通过在空时表示中学习扩散变换器，我们可以在轨迹提示的条件下生成 4D 占据，展示了对驾驶场景的空间和时间分布的理解能力。具有轨迹感知的 4D 生成，OccSora 具有作为自动驾驶决策的世界模拟器的潜力。