With the rapid development of autonomous driving and SLAM technology, the
performance of autonomous systems using multimodal sensors highly relies on
accurate extrinsic calibration. Addressing the need for a convenient,
maintenance-friendly calibration process in any natural environment, this paper
introduces Galibr, a fully automatic targetless LiDAR-camera extrinsic
calibration tool designed for ground vehicle platforms in any natural setting.
The method utilizes the ground planes and edge information from both LiDAR and
camera inputs, streamlining the calibration process. It encompasses two main
steps: an initial pose estimation algorithm based on ground planes (GP-init),
and a refinement phase through edge extraction and matching. Our approach
significantly enhances calibration performance, primarily attributed to our
novel initial pose estimation method, as demonstrated in unstructured natural
environments, including on the KITTI dataset and the KAIST quadruped dataset.

该论文介绍了一种便捷、易于维护的全自动无目标 LiDAR - 相机外参标定工具 Galibr，用于地面车辆平台在任何自然环境下进行标定，通过利用 LiDAR 和相机输入的地面平面和边缘信息，流线化标定过程，该方法包括两个主要步骤：基于地面平面的初始位姿估计算法 (GP-init) 和通过边缘提取和匹配的细化阶段，在 KITTI 数据集和 KAIST 四足数据集等非结构化自然环境中展示了我们的方法明显提升了标定性能，主要归功于我们的初始位姿估计方法。

Galibr：基于地面平面初始化的无目标 LiDAR - 相机外参标定方法

Galibr: Targetless LiDAR-Camera Extrinsic Calibration Method via Ground  Plane Initialization

Object detection with on-board sensors (e.g., lidar, radar, and camera) play
a crucial role in autonomous driving (AD), and these sensors complement each
other in modalities. While crowdsensing may potentially exploit these sensors
(of huge quantity) to derive more comprehensive knowledge, \textit{federated
learning} (FL) appears to be the necessary tool to reach this potential: it
enables autonomous vehicles (AVs) to train machine learning models without
explicitly sharing raw sensory data. However, the multimodal sensors introduce
various data heterogeneity across distributed AVs (e.g., label quantity skews
and varied modalities), posing critical challenges to effective FL. To this
end, we present AutoFed as a heterogeneity-aware FL framework to fully exploit
multimodal sensory data on AVs and thus enable robust AD. Specifically, we
first propose a novel model leveraging pseudo-labeling to avoid mistakenly
treating unlabeled objects as the background. We also propose an
autoencoder-based data imputation method to fill missing data modality (of
certain AVs) with the available ones. To further reconcile the heterogeneity,
we finally present a client selection mechanism exploiting the similarities
among client models to improve both training stability and convergence rate.
Our experiments on benchmark dataset confirm that AutoFed substantially
improves over status quo approaches in both precision and recall, while
demonstrating strong robustness to adverse weather conditions.

AutoFed 是一个异构感知的联邦学习框架，利用众多多模式传感器数据来实现鲁棒的自动驾驶。

AutoFed: 面向异构的联合多模态学习，实现鲁棒自动驾驶

AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving

The popularity of multimodal sensors and the accessibility of the Internet
have brought us a massive amount of unlabeled multimodal data. Since existing
datasets and well-trained models are primarily unimodal, the modality gap
between a unimodal network and unlabeled multimodal data poses an interesting
problem: how to transfer a pre-trained unimodal network to perform the same
task on unlabeled multimodal data? In this work, we propose multimodal
knowledge expansion (MKE), a knowledge distillation-based framework to
effectively utilize multimodal data without requiring labels. Opposite to
traditional knowledge distillation, where the student is designed to be
lightweight and inferior to the teacher, we observe that a multimodal student
model consistently denoises pseudo labels and generalizes better than its
teacher. Extensive experiments on four tasks and different modalities verify
this finding. Furthermore, we connect the mechanism of MKE to semi-supervised
learning and offer both empirical and theoretical explanations to understand
the denoising capability of a multimodal student.

该研究提出了一种新的基于知识蒸馏的多模态知识增强 (MKE) 框架，能够有效利用大量未标记的多模态数据，并将该框架与半监督学习机制联系起来，提出了关于多模态学生模型去噪能力的理论解释。

多模态知识扩展

Multimodal Knowledge Expansion

In this paper, we present RegNet, the first deep convolutional neural network
(CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between
multimodal sensors, exemplified using a scanning LiDAR and a monocular camera.
Compared to existing approaches, RegNet casts all three conventional
calibration steps (feature extraction, feature matching and global regression)
into a single real-time capable CNN. Our method does not require any human
interaction and bridges the gap between classical offline and target-less
online calibration approaches as it provides both a stable initial estimation
as well as a continuous online correction of the extrinsic parameters. During
training we randomly decalibrate our system in order to train RegNet to infer
the correspondence between projected depth measurements and RGB image and
finally regress the extrinsic calibration. Additionally, with an iterative
execution of multiple CNNs, that are trained on different magnitudes of
decalibration, our approach compares favorably to state-of-the-art methods in
terms of a mean calibration error of 0.28 degrees for the rotational and 6 cm
for the translation components even for large decalibrations up to 1.5 m and 20
degrees.

本研究提出了 RegNet，这是第一个使用卷积神经网络（CNN）推断多模式传感器之间的六自由度（DOF）外部校准的方法，与现有方法相比，RegNet 将所有传统校准步骤（特征提取、特征匹配和全局回归）投射到一个单一的实时可行的 CNN 中。