World models are receiving increasing attention in autonomous driving for their ability to predict potential future scenarios. In this paper, we present BEVWorld, a novel approach that tokenizes multimodal sensor inputs into a unified and compact Bird's Eye View (BEV) latent space for environment modeling. The world model consists of two parts: the multi-modal tokenizer and the latent BEV sequence diffusion model. The multi-modal tokenizer first encodes multi-modality information and the decoder is able to reconstruct the latent BEV tokens into LiDAR and image observations by ray-casting rendering in a self-supervised manner. Then the latent BEV sequence diffusion model predicts future scenarios given action tokens as conditions. Experiments demonstrate the effectiveness of BEVWorld in autonomous driving tasks, showcasing its capability in generating future scenes and benefiting downstream tasks such as perception and motion prediction. Code will be available at https://github.com/zympsyche/BevWorld.

通过对多模态传感器输入进行统一而紧凑的鸟瞰视图（BEV）潜在空间编码，BEVWorld 提供了一种新的方法来预测潜在的未来场景，证明了其在自动驾驶任务中生成未来场景的有效性，并对感知和运动预测等下游任务提供了帮助。

BEVWorld：基于统一BEV潜空间的自动驾驶多模态世界模型