The emergence of Multi-Camera 3D Object Detection (MC3D-Det), facilitated by
bird's-eye view (BEV) representation, signifies a notable progression in 3D
object detection. Scaling MC3D-Det training effectively accommodates varied
camera parameters and urban landscapes, paving the way for the MC3D-Det
foundation model. However, the multi-view fusion stage of the MC3D-Det method
relies on the ill-posed monocular perception during training rather than
surround refinement ability, leading to what we term "surround refinement
degradation". To this end, our study presents a weak-to-strong eliciting
framework aimed at enhancing surround refinement while maintaining robust
monocular perception. Specifically, our framework employs weakly tuned experts
trained on distinct subsets, and each is inherently biased toward specific
camera configurations and scenarios. These biased experts can learn the
perception of monocular degeneration, which can help the multi-view fusion
stage to enhance surround refinement abilities. Moreover, a composite
distillation strategy is proposed to integrate the universal knowledge of 2D
foundation models and task-specific information. Finally, for MC3D-Det joint
training, the elaborate dataset merge strategy is designed to solve the problem
of inconsistent camera numbers and camera parameters. We set up a multiple
dataset joint training benchmark for MC3D-Det and adequately evaluated existing
methods. Further, we demonstrate the proposed framework brings a generalized
and significant boost over multiple baselines. Our code is at
https://github.com/EnVision-Research/Scale-BEV.

提出了一种弱到强的激励框架来增强环境细化，同时保持强健的单目感知，通过基于不同子集训练的弱调优专家来学习单目退化的感知，进而改善多视角融合阶段的环境细化能力，并提出了一种综合蒸馏策略来整合 2D 基础模型和任务特定信息的通用知识，最后，为 MC3D-Det 联合训练设计了一个精心制定的数据集合并策略，解决了不一致的摄像头数量和参数问题，此框架普遍提升了多个基准方法效果。

通过弱到强引发实现多摄像头 3D 物体检测的扩展

Scaling Multi-Camera 3D Object Detection through Weak-to-Strong  Eliciting

Detecting objects in 3D space using multiple cameras, known as Multi-Camera
3D Object Detection (MC3D-Det), has gained prominence with the advent of
bird's-eye view (BEV) approaches. However, these methods often struggle when
faced with unfamiliar testing environments due to the lack of diverse training
data encompassing various viewpoints and environments. To address this, we
propose a novel method that aligns 3D detection with 2D camera plane results,
ensuring consistent and accurate detections. Our framework, anchored in
perspective debiasing, helps the learning of features resilient to domain
shifts. In our approach, we render diverse view maps from BEV features and
rectify the perspective bias of these maps, leveraging implicit foreground
volumes to bridge the camera and BEV planes. This two-step process promotes the
learning of perspective- and context-independent features, crucial for accurate
object detection across varying viewpoints, camera parameters and environment
conditions. Notably, our model-agnostic approach preserves the original network
structure without incurring additional inference costs, facilitating seamless
integration across various models and simplifying deployment. Furthermore, we
also show our approach achieves satisfactory results in real data when trained
only with virtual datasets, eliminating the need for real scene annotations.
Experimental results on both Domain Generalization (DG) and Unsupervised Domain
Adaptation (UDA) clearly demonstrate its effectiveness. Our code will be
released.

通过与 2D 相机平面结果对齐的方法，本研究提出了一种新颖的多相机三维目标检测方法，确保持一致且准确的检测结果。该方法通过视角校正，将鸟瞰视图特征渲染为多视图地图，并纠正了这些地图的透视偏差，利用隐含的前景体来连接相机与鸟瞰视图平面。该两步过程促进了视角和上下文无关特征的学习，对于准确地检测不同视角、相机参数和环境条件下的目标至关重要。此外，该方法采用基于模型的不可知的方法，而不需要额外的推理成本和对真实场景注释，从而促进了各种模型的无缝集成和简化部署。实验结果在领域泛化和无监督领域自适应方面明确展示了其有效性。

基于透视矫正的通用多摄像头三维物体检测

Towards Generalizable Multi-Camera 3D Object Detection via Perspective  Debiasing

Single frame data contains finite information which limits the performance of
the existing vision-based multi-camera 3D object detection paradigms. For
fundamentally pushing the performance boundary in this area, a novel paradigm
dubbed BEVDet4D is proposed to lift the scalable BEVDet paradigm from the
spatial-only 3D space to the spatial-temporal 4D space. We upgrade the naive
BEVDet framework with a few modifications just for fusing the feature from the
previous frame with the corresponding one in the current frame. In this way,
with negligible additional computing budget, we enable BEVDet4D to access the
temporal cues by querying and comparing the two candidate features. Beyond
this, we simplify the task of velocity prediction by removing the factors of
ego-motion and time in the learning target. As a result, BEVDet4D with robust
generalization performance reduces the velocity error by up to -62.9%. This
makes the vision-based methods, for the first time, become comparable with
those relied on LiDAR or radar in this aspect. On challenge benchmark nuScenes,
we report a new record of 54.5% NDS with the high-performance configuration
dubbed BEVDet4D-Base, which surpasses the previous leading method BEVDet-Base
by +7.3% NDS. The source code is publicly available for further research at
this https URL .

提出一种名为 BEVDet4D 的新型监测方法，将视觉单帧数据提升至时空 4D 空间，通过对前一帧和当前帧候选特征的对比，使 BEVDet4D 能够访问时间线索，已达到与雷达检测方法相媲美的检测表现