3D object detection based on roadside cameras is an additional way for
autonomous driving to alleviate the challenges of occlusion and short
perception range from vehicle cameras. Previous methods for roadside 3D object
detection mainly focus on modeling the depth or height of objects, neglecting
the stationary of cameras and the characteristic of inter-frame consistency. In
this work, we propose a novel framework, namely MOSE, for MOnocular 3D object
detection with Scene cuEs. The scene cues are the frame-invariant
scene-specific features, which are crucial for object localization and can be
intuitively regarded as the height between the surface of the real road and the
virtual ground plane. In the proposed framework, a scene cue bank is designed
to aggregate scene cues from multiple frames of the same scene with a carefully
designed extrinsic augmentation strategy. Then, a transformer-based decoder
lifts the aggregated scene cues as well as the 3D position embeddings for 3D
object location, which boosts generalization ability in heterologous scenes.
The extensive experiment results on two public benchmarks demonstrate the
state-of-the-art performance of the proposed method, which surpasses the
existing methods by a large margin.

基于 MOSE 框架的道路摄像机 3D 物体检测利用场景线索提高目标定位精度和推广能力，实现了超越现有方法的性能。

MOSE: 基于场景线索增强的基于视觉的路旁 3D 物体检测

MOSE: Boosting Vision-based Roadside 3D Object Detection with Scene Cues

Although the majority of recent autonomous driving systems concentrate on
developing perception methods based on ego-vehicle sensors, there is an
overlooked alternative approach that involves leveraging intelligent roadside
cameras to help extend the ego-vehicle perception ability beyond the visual
range. We discover that most existing monocular 3D object detectors rely on the
ego-vehicle prior assumption that the optical axis of the camera is parallel to
the ground. However, the roadside camera is installed on a pole with a pitched
angle, which makes the existing methods not optimal for roadside scenes. In
this paper, we introduce a novel framework for Roadside Monocular 3D object
detection with ground-aware embeddings, named MonoGAE. Specifically, the ground
plane is a stable and strong prior knowledge due to the fixed installation of
cameras in roadside scenarios. In order to reduce the domain gap between the
ground geometry information and high-dimensional image features, we employ a
supervised training paradigm with a ground plane to predict high-dimensional
ground-aware embeddings. These embeddings are subsequently integrated with
image features through cross-attention mechanisms. Furthermore, to improve the
detector's robustness to the divergences in cameras' installation poses, we
replace the ground plane depth map with a novel pixel-level refined ground
plane equation map. Our approach demonstrates a substantial performance
advantage over all previous monocular 3D object detectors on widely recognized
3D detection benchmarks for roadside cameras. The code and pre-trained models
will be released soon.

该研究提出了一种新的基于智能路边摄像头的道路单目三维物体检测框架 MonoGAE，通过使用地面感知嵌入和交叉注意机制将地面几何信息与图像特征相结合，从而提高自主驾驶系统对道路场景的感知精度。

MonoGAE：具有地面感知的路边单目 3D 物体检测

MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware  Embeddings

While most recent autonomous driving system focuses on developing perception
methods on ego-vehicle sensors, people tend to overlook an alternative approach
to leverage intelligent roadside cameras to extend the perception ability
beyond the visual range. We discover that the state-of-the-art vision-centric
bird's eye view detection methods have inferior performances on roadside
cameras. This is because these methods mainly focus on recovering the depth
regarding the camera center, where the depth difference between the car and the
ground quickly shrinks while the distance increases. In this paper, we propose
a simple yet effective approach, dubbed BEVHeight++, to address this issue. In
essence, we regress the height to the ground to achieve a distance-agnostic
formulation to ease the optimization process of camera-only perception methods.
By incorporating both height and depth encoding techniques, we achieve a more
accurate and robust projection from 2D to BEV spaces. On popular 3D detection
benchmarks of roadside cameras, our method surpasses all previous
vision-centric methods by a significant margin. In terms of the ego-vehicle
scenario, our BEVHeight++ possesses superior over depth-only methods.
Specifically, it yields a notable improvement of +1.9% NDS and +1.1% mAP over
BEVDepth when evaluated on the nuScenes validation set. Moreover, on the
nuScenes test set, our method achieves substantial advancements, with an
increase of +2.8% NDS and +1.7% mAP, respectively.

利用智能路侧摄像头扩展感知能力来解决现有视觉中心的鸟瞰检测方法在路侧摄像头上性能较差的问题。通过回归地面高度并结合高度和深度编码技术，提出了一种简单而有效的方法，名为 BEVHeight++，在 2D 到 BEV 空间的映射中取得了更准确和稳健的结果，超过了其他方法，拥有较好的性能表现。

BEVHeight++：面向稳健的视觉中心化三维目标检测

BEVHeight++: Toward Robust Visual Centric 3D Object Detection

Monocular 3D vehicle localization is an important task in Intelligent
Transportation System (ITS) and Cooperative Vehicle Infrastructure System
(CVIS), which is usually achieved by monocular 3D vehicle detection. However,
depth information cannot be obtained directly by monocular cameras due to the
inherent imaging mechanism, resulting in more challenging monocular 3D tasks.
Most of the current monocular 3D vehicle detection methods leverage 2D
detectors and additional geometric modules, which reduces the efficiency. In
this paper, we propose a 3D vehicle localization network CenterLoc3D for
roadside monocular cameras, which directly predicts centroid and eight vertexes
in image space, and the dimension of 3D bounding boxes without 2D detectors. To
improve the precision of 3D vehicle localization, we propose a weighted-fusion
module and a loss with spatial constraints embedded in CenterLoc3D. Firstly,
the transformation matrix between 2D image space and 3D world space is solved
by camera calibration. Secondly, vehicle type, centroid, eight vertexes, and
the dimension of 3D vehicle bounding boxes are obtained by CenterLoc3D.
Finally, centroid in 3D world space can be obtained by camera calibration and
CenterLoc3D for 3D vehicle localization. To the best of our knowledge, this is
the first application of 3D vehicle localization for roadside monocular
cameras. Hence, we also propose a benchmark for this application including a
dataset (SVLD-3D), an annotation tool (LabelImg-3D), and evaluation metrics.
Through experimental validation, the proposed method achieves high accuracy and
real-time performance. (limited words, please see the article for more details)

本文提出了一种针对单目相机的 3D 车辆定位网络 CenterLoc3D，通过解决摄像头标定和使用空间约束嵌入到 CenterLoc3D 的损失函数实现了用于路侧单目摄像头的 3D 车辆定位，同时提出了 SVLD-3D 数据集、标注工具和评估指标。经实验验证，所提出的方法具有高精度和实时性。