Road surface conditions, especially geometry profiles, enormously affect
driving performance of autonomous vehicles. Vision-based online road
reconstruction promisingly captures road information in advance. Existing
solutions like monocular depth estimation and stereo matching suffer from
modest performance. The recent technique of Bird's-Eye-View (BEV) perception
provides immense potential to more reliable and accurate reconstruction. This
paper uniformly proposes two simple yet effective models for road elevation
reconstruction in BEV named RoadBEV-mono and RoadBEV-stereo, which estimate
road elevation with monocular and stereo images, respectively. The former
directly fits elevation values based on voxel features queried from image view,
while the latter efficiently recognizes road elevation patterns based on BEV
volume representing discrepancy between left and right voxel features.
Insightful analyses reveal their consistence and difference with perspective
view. Experiments on real-world dataset verify the models' effectiveness and
superiority. Elevation errors of RoadBEV-mono and RoadBEV-stereo achieve 1.83cm
and 0.56cm, respectively. The estimation performance improves by 50\% in BEV
based on monocular image. Our models are promising for practical applications,
providing valuable references for vision-based BEV perception in autonomous
driving. The code is released at this https URL

本研究提出了两种简单而有效的模型（RoadBEV-mono 和 RoadBEV-stereo），分别使用单目和立体图像估计道路高程，在 Bird's-Eye-View 感知中实现对道路的可靠和准确重建。经实验证明，RoadBEV-mono 和 RoadBEV-stereo 的高程误差分别为 1.83cm 和 0.56cm，在单目图像的基础上，估计性能提高了 50％，这些模型在自动驾驶中具有实际应用的潜力，并为基于视觉的 Bird's-Eye-View 感知提供了有价值的参考。

RoadBEV：鸟瞰视角下的道路表面重建

RoadBEV: Road Surface Reconstruction in Bird's Eye View

Camera-based Bird's-Eye-View (BEV) perception often struggles between
adopting 3D-to-2D or 2D-to-3D view transformation (VT). The 3D-to-2D VT
typically employs resource intensive Transformer to establish robust
correspondences between 3D and 2D feature, while the 2D-to-3D VT utilizes the
Lift-Splat-Shoot (LSS) pipeline for real-time application, potentially missing
distant information. To address these limitations, we propose DualBEV, a
unified framework that utilizes a shared CNN-based feature transformation
incorporating three probabilistic measurements for both strategies. By
considering dual-view correspondences in one-stage, DualBEV effectively bridges
the gap between these strategies, harnessing their individual strengths. Our
method achieves state-of-the-art performance without Transformer, delivering
comparable efficiency to the LSS approach, with 55.2% mAP and 63.4% NDS on the
nuScenes test set. Code will be released at
this https URL

Camera-based Bird's-Eye-View perception, DualBEV, eliminates the need for a resource-intensive Transformer by proposing a unified framework utilizing a shared CNN-based feature transformation, resulting in state-of-the-art performance and comparable efficiency to the Lift-Splat-Shoot approach.

DualBEV: CNN 是视角变换中所需的全部

DualBEV: CNN is All You Need in View Transformation

Recently, the pure camera-based Bird's-Eye-View (BEV) perception provides a
feasible solution for economical autonomous driving. However, the existing
BEV-based multi-view 3D detectors generally transform all image features into
BEV features, without considering the problem that the large proportion of
background information may submerge the object information. In this paper, we
propose Semantic-Aware BEV Pooling (SA-BEVPool), which can filter out
background information according to the semantic segmentation of image features
and transform image features into semantic-aware BEV features. Accordingly, we
propose BEV-Paste, an effective data augmentation strategy that closely matches
with semantic-aware BEV feature. In addition, we design a Multi-Scale
Cross-Task (MSCT) head, which combines task-specific and cross-task information
to predict depth distribution and semantic segmentation more accurately,
further improving the quality of semantic-aware BEV feature. Finally, we
integrate the above modules into a novel multi-view 3D object detection
framework, namely SA-BEV. Experiments on nuScenes show that SA-BEV achieves
state-of-the-art performance. Code has been available at
this https URL

本文介绍了一种 Semantic-Aware BEV Pooling (SA-BEVPool) 的方法，通过语义分割图像特征来过滤背景信息，并将图像特征转化为语义感知的 BEV 特征。同时，提出了一种与语义感知 BEV 特征相匹配的有效数据增强策略 BEV-Paste。此外，设计了一个多尺度交叉任务头（MSCT），结合特定任务和交叉任务的信息来更准确地预测深度分布和语义分割，进一步提高语义感知 BEV 特征的质量。最后，将这些模块整合到一个新的多视角 3D 目标检测框架 SA-BEV 中，在 nuScenes 数据集上达到了最先进的性能。

SA-BEV: 多视角 3D 目标检测中生成语义感知鸟瞰特征

SA-BEV: Generating Semantic-Aware Bird's-Eye-View Feature for Multi-view  3D Object Detection

Learning powerful representations in bird's-eye-view (BEV) for perception
tasks is trending and drawing extensive attention both from industry and
academia. Conventional approaches for most autonomous driving algorithms
perform detection, segmentation, tracking, etc., in a front or perspective
view. As sensor configurations get more complex, integrating multi-source
information from different sensors and representing features in a unified view
come of vital importance. BEV perception inherits several advantages, as
representing surrounding scenes in BEV is intuitive and fusion-friendly; and
representing objects in BEV is most desirable for subsequent modules as in
planning and/or control. The core problems for BEV perception lie in (a) how to
reconstruct the lost 3D information via view transformation from perspective
view to BEV; (b) how to acquire ground truth annotations in BEV grid; (c) how
to formulate the pipeline to incorporate features from different sources and
views; and (d) how to adapt and generalize algorithms as sensor configurations
vary across different scenarios. In this survey, we review the most recent work
on BEV perception and provide an in-depth analysis of different solutions.
Moreover, several systematic designs of BEV approach from the industry are
depicted as well. Furthermore, we introduce a full suite of practical guidebook
to improve the performance of BEV perception tasks, including camera, LiDAR and
fusion inputs. At last, we point out the future research directions in this
area. We hope this report would shed some light on the community and encourage
more research effort on BEV perception. We keep an active repository to collect
the most recent work and provide a toolbox for bag of tricks at
this https URL

本文综述了关于 Bird's-eye-view（BEV）感知的最新研究工作，探讨了多传感器融合、BEV 视角下物体检测与定位等关键问题，并介绍了一系列实用的指南和工具来提高 BEV 任务的性能，最后指出了该领域未来的研究方向。