Due to the lack of depth cues in images, multi-frame inputs are important for
the success of vision-based perception, prediction, and planning in autonomous
driving. Observations from different angles enable the recovery of 3D object
states from 2D image inputs if we can identify the same instance in different
input frames. However, the dynamic nature of autonomous driving scenes leads to
significant changes in the appearance and shape of each instance captured by
the camera at different time steps. To this end, we propose a novel contrastive
learning algorithm, Cohere3D, to learn coherent instance representations in a
long-term input sequence robust to the change in distance and perspective. The
learned representation aids in instance-level correspondence across multiple
input frames in downstream tasks. In the pretraining stage, the raw point
clouds from LiDAR sensors are utilized to construct the long-term temporal
correspondence for each instance, which serves as guidance for the extraction
of instance-level representation from the vision-based bird's eye-view (BEV)
feature map. Cohere3D encourages a consistent representation for the same
instance at different frames but distinguishes between representations of
different instances. We evaluate our algorithm by finetuning the pretrained
model on various downstream perception, prediction, and planning tasks. Results
show a notable improvement in both data efficiency and task performance.

为了克服图像中缺乏深度线索的问题，本文提出了一种新的对比学习算法 Cohere3D，用于在长期输入序列中学习一致的实例表示，从而在自动驾驶的下游任务中实现多个输入帧的实例级对应。通过预训练阶段中从 LiDAR 传感器中的原始点云构建长期时序对应来辅助从基于视觉的鸟瞰特征图中提取实例级表示。实验证明该算法在数据效率和任务性能方面都取得显著改进。

Cohere3D：利用时间一致性进行无监督视觉自动驾驶的表示学习

Cohere3D: Exploiting Temporal Coherence for Unsupervised Representation  Learning of Vision-based Autonomous Driving

In recent years, camera-based 3D object detection has gained widespread
attention for its ability to achieve high performance with low computational
cost. However, the robustness of these methods to adversarial attacks has not
been thoroughly examined. In this study, we conduct the first comprehensive
investigation of the robustness of leading camera-based 3D object detection
methods under various adversarial conditions. Our experiments reveal five
interesting findings: (a) the use of accurate depth estimation effectively
improves robustness; (b) depth-estimation-free approaches do not show superior
robustness; (c) bird's-eye-view-based representations exhibit greater
robustness against localization attacks; (d) incorporating multi-frame benign
inputs can effectively mitigate adversarial attacks; and (e) addressing
long-tail problems can enhance robustness. We hope our work can provide
guidance for the design of future camera-based object detection modules with
improved adversarial robustness.

该研究考察了基于相机的 3D 物体检测方法在不同对抗条件下的鲁棒性，并发现了提高鲁棒性的关键因素，包括准确的深度估计、鸟瞰图表示法、多帧输入等，为今后设计具有改进的鲁棒性的基于相机的物体检测模块提供指导。