Self-supervised learning (SSL) for RGB images has achieved significant
success, yet there is still limited research on SSL for infrared images,
primarily due to three prominent challenges: 1) the lack of a suitable
large-scale infrared pre-training dataset, 2) the distinctiveness of non-iconic
infrared images rendering common pre-training tasks like masked image modeling
(MIM) less effective, and 3) the scarcity of fine-grained textures making it
particularly challenging to learn general image features. To address these
issues, we construct a Multi-Scene Infrared Pre-training (MSIP) dataset
comprising 178,756 images, and introduce object-sensitive random RoI cropping,
an image preprocessing method, to tackle the challenge posed by non-iconic
images. To alleviate the impact of weak textures on feature learning, we
propose a pre-training paradigm called Pre-training with ADapter (PAD), which
uses adapters to learn domain-specific features while freezing parameters
pre-trained on ImageNet to retain the general feature extraction capability.
This new paradigm is applicable to any transformer-based SSL method.
Furthermore, to achieve more flexible coordination between pre-trained and
newly-learned features in different layers and patches, a patchwise-scale
adapter with dynamically learnable scale factors is introduced. Extensive
experiments on three downstream tasks show that PAD, with only 1.23M
pre-trainable parameters, outperforms other baseline paradigms including
continual full pre-training on MSIP. Our code and dataset are available at
this https URL

该研究介绍了自主学习在红外图像领域的应用，提供了一个大规模红外预训练数据集，通过对象敏感的随机 RoI 裁剪方法和面向纹理缺失的图像预处理方法解决了传统方法在红外图像上的挑战，并引入可动态学习尺度因子的分块适配器来实现预训练特征和新特征之间的灵活协调。实验证明，在三个下游任务上，该方法优于其他基线模型，仅使用了 1.23M 可预训练参数。