In this paper, we introduce the first large-scale video prediction model in
the autonomous driving discipline. To eliminate the restriction of high-cost
data collection and empower the generalization ability of our model, we acquire
massive data from the web and pair it with diverse and high-quality text
descriptions. The resultant dataset accumulates over 2000 hours of driving
videos, spanning areas all over the world with diverse weather conditions and
traffic scenarios. Inheriting the merits from recent latent diffusion models,
our model, dubbed GenAD, handles the challenging dynamics in driving scenes
with novel temporal reasoning blocks. We showcase that it can generalize to
various unseen driving datasets in a zero-shot manner, surpassing general or
driving-specific video prediction counterparts. Furthermore, GenAD can be
adapted into an action-conditioned prediction model or a motion planner,
holding great potential for real-world driving applications.

本文介绍了自动驾驶学科中第一个大规模视频预测模型。通过从网络获取大量数据并与多样且高质量的文本描述配对，我们消除了高成本数据收集的限制并增强了模型的泛化能力。该数据集积累了超过 2000 小时的行驶视频，涵盖了全球各地不同的气候条件和交通场景。我们的模型名为 GenAD，并继承了最近潜变扩散模型的优点，利用新颖的时间推理块处理行驶场景中具有挑战性的动态。我们展示了它可以以零 - shot 方式泛化到各种未见过的行驶数据集，超过了一般或专门针对行车的视频预测模型。此外，GenAD 可以适应动作条件化预测模型或运动规划器，具有在实际行驶应用中巨大潜力。