This paper focuses on the area of RGB(visible)-NIR(near-infrared)
cross-modality image registration, which is crucial for many downstream vision
tasks to fully leverage the complementary information present in visible and
infrared images. In this field, researchers face two primary challenges - the
absence of a correctly-annotated benchmark with viewpoint variations for
evaluating RGB-NIR cross-modality registration methods and the problem of
inconsistent local features caused by the appearance discrepancy between
RGB-NIR cross-modality images. To address these challenges, we first present
the RGB-NIR Image Registration (RGB-NIR-IRegis) benchmark, which, for the first
time, enables fair and comprehensive evaluations for the task of RGB-NIR
cross-modality image registration. Evaluations of previous methods highlight
the significant challenges posed by our RGB-NIR-IRegis benchmark, especially on
RGB-NIR image pairs with viewpoint variations. To analyze the causes of the
unsatisfying performance, we then design several metrics to reveal the toxic
impact of inconsistent local features between visible and infrared images on
the model performance. This further motivates us to develop a baseline method
named Semantic Guidance Transformer (SGFormer), which utilizes high-level
semantic guidance to mitigate the negative impact of local inconsistent
features. Despite the simplicity of our motivation, extensive experimental
results show the effectiveness of our method.

本研究主要探讨可见光与近红外光交叉模态图像配准的问题，提出 RGB-NIR-IRegis 基准评估方法并研究视角变化对 RGB-NIR 图像配准的挑战，设计了度量标准分析可见光和红外光图像不一致局部特征对模型性能的消极影响，并提出了使用高层语义引导的 SGFormer 基线方法，实验证明该方法的有效性。