Conditional Imitation learning is a common and effective approach to train
autonomous driving agents. However, two issues limit the full potential of this
approach: (i) the inertia problem, a special case of causal confusion where the
agent mistakenly correlates low speed with no acceleration, and (ii) low
correlation between offline and online performance due to the accumulation of
small errors that brings the agent in a previously unseen state. Both issues
are critical for state-aware models, yet informing the driving agent of its
internal state as well as the state of the environment is of crucial
importance. In this paper we propose a multi-task learning agent based on a
multi-stage vision transformer with state token propagation. We feed the state
of the vehicle along with the representation of the environment as a special
token of the transformer and propagate it throughout the network. This allows
us to tackle the aforementioned issues from different angles: guiding the
driving policy with learned stop/go information, performing data augmentation
directly on the state of the vehicle and visually explaining the model's
decisions. We report a drastic decrease in inertia and a high correlation
between offline and online metrics.

本文提出了一种基于多阶段视觉 Transformer 的多任务学习代理，通过传播车辆状态和环境表示作为 Transformer 的特殊令牌，并从不同角度解决了惯性和离线与在线性能之间的低相关性问题。在实验中，我们观察到惯性显著减少，并且离线和在线指标之间有很高的相关性。

解决自动驾驶中状态感知模仿学习的局限性

Addressing Limitations of State-Aware Imitation Learning for Autonomous  Driving

Conditional imitation learning (CIL) trains deep neural networks, in an
end-to-end manner, to mimic human driving. This approach has demonstrated
suitable vehicle control when following roads, avoiding obstacles, or taking
specific turns at intersections to reach a destination. Unfortunately,
performance dramatically decreases when deployed to unseen environments and is
inconsistent against varying weather conditions. Most importantly, the current
CIL fails to avoid static road blockages. In this work, we propose a solution
to those deficiencies. First, we fuse the laser scanner with the regular camera
streams, at the features level, to overcome the generalization and consistency
challenges. Second, we introduce a new efficient Occupancy Grid Mapping (OGM)
method along with new algorithms for road blockages avoidance and global route
planning. Consequently, our proposed method dynamically detects partial and
full road blockages, and guides the controlled vehicle to another route to
reach the destination. Following the original CIL work, we demonstrated the
effectiveness of our proposal on CARLA simulator urban driving benchmark. Our
experiments showed that our model improved consistency against weather
conditions by four times and autonomous driving success rate generalization by
52%. Furthermore, our global route planner improved the driving success rate by
37%. Our proposed road blockages avoidance algorithm improved the driving
success rate by 27%. Finally, the average kilometers traveled before a
collision with a static object increased by 1.5 times. The main source code can
be reached at this https URL.

通过将激光扫描仪和常规相机流在特征级别上融合，以克服泛化和一致性挑战，并引入新的高效实用的栅格占据映射方法及其相关算法，从而动态检测道路的不同障碍，并通过指南针指引可控车辆到另一条路线上以到达目的地。该方法在 CARLA 模拟器城市驾驶基准测试中得到了证明，具有显著效果提升。

自主驾驶的动态条件模仿学习

Dynamic Conditional Imitation Learning for Autonomous Driving

Widespread adoption of self-driving cars will depend not only on their safety
but largely on their ability to interact with human users. Just like human
drivers, self-driving cars will be expected to understand and safely follow
natural-language directions that suddenly alter the pre-planned route according
to user's preference or in presence of ambiguities, particularly in locations
with poor or outdated map coverage. To this end, we propose a language-grounded
driving agent implementing a hierarchical policy using recurrent layers and
gated attention. The hierarchical approach enables us to reason both in terms
of high-level language instructions describing long time horizons and
low-level, complex, continuous state/action spaces required for real-time
control of a self-driving car. We train our policy with conditional imitation
learning from realistic language data collected from human drivers and
navigators. Through quantitative and interactive experiments within the CARLA
framework, we show that our model can successfully interpret language
instructions and follow them safely, even when generalizing to previously
unseen environments. Code and video are available at
this https URL

该研究介绍了一种基于语言的驾驶智能体，它使用递归层和门控注意力实现分层策略，通过条件模仿学习培训策略，并能够成功地解释语言指令并安全地跟随它们，甚至在以前未见过的环境中进行泛化。