World models can foresee the outcomes of different actions, which is of
paramount importance for autonomous driving. Nevertheless, existing driving
world models still have limitations in generalization to unseen environments,
prediction fidelity of critical details, and action controllability for
flexible application. In this paper, we present Vista, a generalizable driving
world model with high fidelity and versatile controllability. Based on a
systematic diagnosis of existing methods, we introduce several key ingredients
to address these limitations. To accurately predict real-world dynamics at high
resolution, we propose two novel losses to promote the learning of moving
instances and structural information. We also devise an effective latent
replacement approach to inject historical frames as priors for coherent
long-horizon rollouts. For action controllability, we incorporate a versatile
set of controls from high-level intentions (command, goal point) to low-level
maneuvers (trajectory, angle, and speed) through an efficient learning
strategy. After large-scale training, the capabilities of Vista can seamlessly
generalize to different scenarios. Extensive experiments on multiple datasets
show that Vista outperforms the most advanced general-purpose video generator
in over 70% of comparisons and surpasses the best-performing driving world
model by 55% in FID and 27% in FVD. Moreover, for the first time, we utilize
the capacity of Vista itself to establish a generalizable reward for real-world
action evaluation without accessing the ground truth actions.

通过系统诊断现有方法的不足，我们提出了具有高保真度和灵活可控性的可扩展驾驶世界模型 Vista，利用新的损失函数促进移动实例和结构信息的学习，并通过有效的潜在替代方法注入历史帧作为前提进行连贯的长程预测。此外，我们利用 Vista 本身的能力首次建立了一种对真实世界动作评估的通用奖励。