How should we integrate representations from complementary sensors for
autonomous driving? Geometry-based fusion has shown promise for perception
(e.g. object detection, motion forecasting). However, in the context of
end-to-end driving, we find that imitation learning based on existing sensor
fusion methods underperforms in complex driving scenarios with a high density
of dynamic agents. Therefore, we propose TransFuser, a mechanism to integrate
image and LiDAR representations using self-attention. Our approach uses
transformer modules at multiple resolutions to fuse perspective view and bird's
eye view feature maps. We experimentally validate its efficacy on a challenging
new benchmark with long routes and dense traffic, as well as the official
leaderboard of the CARLA urban driving simulator. At the time of submission,
TransFuser outperforms all prior work on the CARLA leaderboard in terms of
driving score by a large margin. Compared to geometry-based fusion, TransFuser
reduces the average collisions per kilometer by 48%.

该研究介绍了 TransFuser - 一种使用自我注意力机制将图像和雷达的信息进行融合，以提高自动驾驶中辨识物体和行驶路径的性能，实验证明该方法的表现在 CARLA 排行榜上优于以往所有的算法。

TransFuser：基于 Transformer 传感器融合的自动驾驶模仿

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

In this paper, we present a system to train driving policies from experiences
collected not just from the ego-vehicle, but all vehicles that it observes.
This system uses the behaviors of other agents to create more diverse driving
scenarios without collecting additional data. The main difficulty in learning
from other vehicles is that there is no sensor information. We use a set of
supervisory tasks to learn an intermediate representation that is invariant to
the viewpoint of the controlling vehicle. This not only provides a richer
signal at training time but also allows more complex reasoning during
inference. Learning how all vehicles drive helps predict their behavior at test
time and can avoid collisions. We evaluate this system in closed-loop driving
simulations. Our system outperforms all prior methods on the public CARLA
Leaderboard by a wide margin, improving driving score by 25 and route
completion rate by 24 points. Our method won the 2021 CARLA Autonomous Driving
challenge. Code and data are available at this https URL

本文提出了一种通过其他车辆的行为来训练自动驾驶策略的系统，使用监督任务来学习不受视角限制的中间表示，进而预测其他车辆的行为以避免碰撞，并在 CARLA Leaderboard 上取得了出色的成绩。