Image-to-point cloud (I2P) registration is a fundamental task in the fields
of robot navigation and mobile mapping. Existing I2P registration works
estimate correspondences at the point-to-pixel level, neglecting the global
alignment. However, I2P matching without high-level guidance from global
constraints may converge to the local optimum easily. To solve the problem,
this paper proposes CoFiI2P, a novel I2P registration network that extracts
correspondences in a coarse-to-fine manner for the global optimal solution.
First, the image and point cloud are fed into a Siamese encoder-decoder network
for hierarchical feature extraction. Then, a coarse-to-fine matching module is
designed to exploit features and establish resilient feature correspondences.
Specifically, in the coarse matching block, a novel I2P transformer module is
employed to capture the homogeneous and heterogeneous global information from
image and point cloud. With the discriminate descriptors, coarse
super-point-to-super-pixel matching pairs are estimated. In the fine matching
module, point-to-pixel pairs are established with the
super-point-to-super-pixel correspondence supervision. Finally, based on
matching pairs, the transform matrix is estimated with the EPnP-RANSAC
algorithm. Extensive experiments conducted on the KITTI dataset have
demonstrated that CoFiI2P achieves a relative rotation error (RRE) of 2.25
degrees and a relative translation error (RTE) of 0.61 meters. These results
represent a significant improvement of 14% in RRE and 52% in RTE compared to
the current state-of-the-art (SOTA) method. The demo video for the experiments
is available at this https URL The source code will be public at
this https URL

该研究论文介绍了 CoFiI2P，一种新颖的图像到点云注册网络，通过层次化特征提取和鲁棒特征对应的建立，以获得全局最优解决方案，并在 KITTI 数据集上展示了相对旋转误差（RRE）为 2.25 度和相对平移误差（RTE）为 0.61 米的显著提升。