The well-established modular autonomous driving system is decoupled into
different standalone tasks, e.g. perception, prediction and planning, suffering
from information loss and error accumulation across modules. In contrast,
end-to-end paradigms unify multi-tasks into a fully differentiable framework,
allowing for optimization in a planning-oriented spirit. Despite the great
potential of end-to-end paradigms, both the performance and efficiency of
existing methods are not satisfactory, particularly in terms of planning
safety. We attribute this to the computationally expensive BEV (bird's eye
view) features and the straightforward design for prediction and planning. To
this end, we explore the sparse representation and review the task design for
end-to-end autonomous driving, proposing a new paradigm named SparseDrive.
Concretely, SparseDrive consists of a symmetric sparse perception module and a
parallel motion planner. The sparse perception module unifies detection,
tracking and online mapping with a symmetric model architecture, learning a
fully sparse representation of the driving scene. For motion prediction and
planning, we review the great similarity between these two tasks, leading to a
parallel design for motion planner. Based on this parallel design, which models
planning as a multi-modal problem, we propose a hierarchical planning selection
strategy , which incorporates a collision-aware rescore module, to select a
rational and safe trajectory as the final planning output. With such effective
designs, SparseDrive surpasses previous state-of-the-arts by a large margin in
performance of all tasks, while achieving much higher training and inference
efficiency. Code will be avaliable at this https URL for
facilitating future research.

稀疏驾驶 (SparseDrive) 是一种新的端到端自动驾驶范例，通过使用稀疏表示和并行动作规划，结合多模式问题建模和冲突感知重评分模块，实现了任务性能和效率的显著提升。

SparseDrive: 通过稀疏场景表示实现端到端自主驾驶

SparseDrive: End-to-End Autonomous Driving via Sparse Scene  Representation

End-to-End paradigms use a unified framework to implement multi-tasks in an
autonomous driving system. Despite simplicity and clarity, the performance of
end-to-end autonomous driving methods on sub-tasks is still far behind the
single-task methods. Meanwhile, the widely used dense BEV features in previous
end-to-end methods make it costly to extend to more modalities or tasks. In
this paper, we propose a Sparse query-centric paradigm for end-to-end
Autonomous Driving (SparseAD), where the sparse queries completely represent
the whole driving scenario across space, time and tasks without any dense BEV
representation. Concretely, we design a unified sparse architecture for
perception tasks including detection, tracking, and online mapping. Moreover,
we revisit motion prediction and planning, and devise a more justifiable motion
planner framework. On the challenging nuScenes dataset, SparseAD achieves SOTA
full-task performance among end-to-end methods and significantly narrows the
performance gap between end-to-end paradigms and single-task methods. Codes
will be released soon.

在此论文中，我们提出了一种以稀疏查询为中心的端到端自动驾驶（SparseAD）范式，其中稀疏查询完全代表了整个驾驶场景在空间、时间和任务上的具体表征。通过针对感知任务设计了统一的稀疏架构，包括检测、跟踪和在线地图绘制。此外，我们重新考虑了运动预测和规划，并设计了更具合理性的运动规划器框架。在具有挑战性的 nuScenes 数据集上，SparseAD 在端到端方法中实现了 SOTA 全任务性能，显著缩小了端到端范式和单任务方法之间的性能差距。具体代码即将发布。

SparseAD：高效端到端自动驾驶的稀疏查询中心范式

SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End  Autonomous Driving

End-to-end paradigms significantly improve the accuracy of various
deep-learning-based computer vision models. To this end, tasks like object
detection have been upgraded by replacing non-end-to-end components, such as
removing non-maximum suppression by training with a set loss based on bipartite
matching. However, such an upgrade is not applicable to instance segmentation,
due to its significantly higher output dimensions compared to object detection.
In this paper, we propose an instance segmentation Transformer, termed ISTR,
which is the first end-to-end framework of its kind. ISTR predicts
low-dimensional mask embeddings, and matches them with ground truth mask
embeddings for the set loss. Besides, ISTR concurrently conducts detection and
segmentation with a recurrent refinement strategy, which provides a new way to
achieve instance segmentation compared to the existing top-down and bottom-up
frameworks. Benefiting from the proposed end-to-end mechanism, ISTR
demonstrates state-of-the-art performance even with approximation-based
suboptimal embeddings. Specifically, ISTR obtains a 46.8/38.6 box/mask AP using
ResNet50-FPN, and a 48.1/39.9 box/mask AP using ResNet101-FPN, on the MS COCO
dataset. Quantitative and qualitative results reveal the promising potential of
ISTR as a solid baseline for instance-level recognition. Code has been made
available at: this https URL

本文提出了一种名为 ISTR 的实例分割变压器，利用低维掩模嵌入进行实例分割，采用递归细化策略实现检测和分割并行，实现了端到端框架。在 MS COCO 数据集上，基于 ResNet50-FPN 和 ResNet101-FPN 的 ISTR 分别获得了 46.8 / 38.6 和 48.1 / 39.9 的盒子 / 掩码 AP，并取得了最先进的性能。