We present LidarDM, a novel LiDAR generative model capable of producing realistic, layout-aware, physically plausible, and temporally coherent LiDAR videos. LidarDM stands out with two unprecedented capabilities in LiDAR generative modeling: (i) LiDAR generation guided by driving scenarios, offering significant potential for autonomous driving simulations, and (ii) 4D LiDAR point cloud generation, enabling the creation of realistic and temporally coherent sequences. At the heart of our model is a novel integrated 4D world generation framework. Specifically, we employ latent diffusion models to generate the 3D scene, combine it with dynamic actors to form the underlying 4D world, and subsequently produce realistic sensory observations within this virtual environment. Our experiments indicate that our approach outperforms competing algorithms in realism, temporal coherency, and layout consistency. We additionally show that LidarDM can be used as a generative world model simulator for training and testing perception models.

LidarDM是一种新颖的LiDAR生成模型，能够生成逼真、具有布局意识、物理合理和时间连贯的LiDAR视频。该模型具有两个前所未有的能力：根据驾驶场景进行LiDAR生成，为自动驾驶模拟提供了重要潜力，并且能够生成4D LiDAR点云，实现逼真且时间连贯的序列。我们的实验表明，我们的方法在逼真性、时间连贯性和布局一致性方面优于竞争算法。此外，我们还展示了LidarDM可用作生成世界模型的模拟器，用于训练和测试感知模型。

LidarDM：生成世界中的LiDAR仿真