We introduce a novel architecture, UniCal, for Camera-to-LiDAR (C2L)
extrinsic calibration which leverages self-attention mechanisms through a
Transformer-based backbone network to infer the 6-degree of freedom (DoF)
relative transformation between the sensors. Unlike previous methods, UniCal
performs an early fusion of the input camera and LiDAR data by aggregating
camera image channels and LiDAR mappings into a multi-channel unified
representation before extracting their features jointly with a single-branch
architecture. This single-branch architecture makes UniCal lightweight, which
is desirable in applications with restrained resources such as autonomous
driving. Through experiments, we show that UniCal achieves state-of-the-art
results compared to existing methods. We also show that through transfer
learning, weights learned on the calibration task can be applied to a
calibration validation task without re-training the backbone.

我们介绍了一种新颖的架构 UniCal，用于相机和 LiDAR 之间的外部标定，通过基于 Transformer 的骨干网络利用自注意机制来推断传感器之间的 6 自由度相对变换。与以往的方法不同，UniCal 通过将相机图像通道和 LiDAR 映射聚合到一个多通道统一表示中，再使用单分支架构联合提取特征，从而实现了早期融合。通过实验证明，我们证明 UniCal 相较于现有方法实现了最先进的结果。同时，我们还展示通过迁移学习，可将在标定任务中学习到的权重应用于标定验证任务而无需重新训练骨干网络。

UniCal: 一种基于单支 Transformer 的相机 - 激光雷达标定与验证模型

UniCal: a Single-Branch Transformer-Based Model for Camera-to-LiDAR  Calibration and Validation

Modelling long-range dependencies is critical for scene understanding tasks
in computer vision. Although convolution neural networks (CNNs) have excelled
in many vision tasks, they are still limited in capturing long-range structured
relationships as they typically consist of layers of local kernels. A
fully-connected graph, such as the self-attention operation in Transformers, is
beneficial for such modelling, however, its computational overhead is
prohibitive. In this paper, we propose a dynamic graph message passing network,
that significantly reduces the computational complexity compared to related
works modelling a fully-connected graph. This is achieved by adaptively
sampling nodes in the graph, conditioned on the input, for message passing.
Based on the sampled nodes, we dynamically predict node-dependent filter
weights and the affinity matrix for propagating information between them. This
formulation allows us to design a self-attention module, and more importantly a
new Transformer-based backbone network, that we use for both image
classification pretraining, and for addressing various downstream tasks (object
detection, instance and semantic segmentation). Using this model, we show
significant improvements with respect to strong, state-of-the-art baselines on
four different tasks. Our approach also outperforms fully-connected graphs
while using substantially fewer floating-point operations and parameters. Code
and models will be made publicly available at
this https URL

本论文提出了一种动态图消息传递网络，用于进行长程依赖性建模，以用于图像识别。该网络采用自适应抽样节点的方法，在传递信息时动态地预测节点相关过滤器权重和关联矩阵，以实现对自我注意机制的设计。研究结果表明，基于该模型的 Transformer 骨干网络在图像分类和物体检测等四种不同任务上，相对于现有的最先进技术，能够显著提升性能，同时性能优于完全连接的图并使用更少的浮点运算和参数。