State-of-the-art approaches for autonomous driving integrate multiple
sub-tasks of the overall driving task into a single pipeline that can be
trained in an end-to-end fashion by passing latent representations between the
different modules. In contrast to previous approaches that rely on a unified
grid to represent the belief state of the scene, we propose dedicated
representations to disentangle dynamic agents and static scene elements. This
allows us to explicitly compensate for the effect of both ego and object motion
between consecutive time steps and to flexibly propagate the belief state
through time. Furthermore, dynamic objects can not only attend to the input
camera images, but also directly benefit from the inferred static scene
structure via a novel dynamic-static cross-attention. Extensive experiments on
the challenging nuScenes benchmark demonstrate the benefits of the proposed
dual-stream design, especially for modelling highly dynamic agents in the
scene, and highlight the improved temporal consistency of our approach. Our
method titled DualAD not only outperforms independently trained single-task
networks, but also improves over previous state-of-the-art end-to-end models by
a large margin on all tasks along the functional chain of driving.

自动驾驶的最新方法将整个驾驶任务的多个子任务整合成一个单一的流水线，可以通过在不同模块之间传递潜在表示进行端到端的训练。与之前依赖统一网格表示场景置信状态的方法相比，我们提出了专用的表示方法来解耦动态代理和静态场景元素，从而明确补偿相邻时间步之间的自我和对象运动的影响，并在时间上灵活地传播置信状态。此外，动态对象不仅可以关注输入的摄像头图像，还可以通过新颖的动态 - 静态交叉注意力直接从推断的静态场景结构中受益。对具有挑战性的 nuScenes 基准的大量实验证明了我们所提出的双流设计的优势，尤其是用于对场景中高动态代理建模，并凸显了我们方法的改进时间一致性。我们的方法 DualAD 不仅胜过独立训练的单任务网络，还在驾驶的功能链上的所有任务上大幅改进了之前最先进的端到端模型。