The commonly adopted detect-then-match approach to registration finds
difficulties in the cross-modality cases due to the incompatible keypoint
detection and inconsistent feature description. We propose, 2D3D-MATR, a
detection-free method for accurate and robust registration between images and
point clouds. Our method adopts a coarse-to-fine pipeline where it first
computes coarse correspondences between downsampled patches of the input image
and the point cloud and then extends them to form dense correspondences between
pixels and points within the patch region. The coarse-level patch matching is
based on transformer which jointly learns global contextual constraints with
self-attention and cross-modality correlations with cross-attention. To resolve
the scale ambiguity in patch matching, we construct a multi-scale pyramid for
each image patch and learn to find for each point patch the best matching image
patch at a proper resolution level. Extensive experiments on two public
benchmarks demonstrate that 2D3D-MATR outperforms the previous state-of-the-art
P2-Net by around $20$ percentage points on inlier ratio and over $10$ points on
registration recall. Our code and models are available at
https://github.com/minhaolee/2D3DMATR.

采用 2D3D-MATR 方法，我们提出了一种无需检测的方法，用于图像和点云之间的准确且鲁棒的配准。该方法在粗匹配的基础上，通过学习全局上下文约束和跨模态相关性，通过 transformer 实现多尺度金字塔和图像块焦点学习，解决了尺度不确定性问题，从而获得了比之前的最优模型 P2-Net 更高的配准率和稳定度。