Vision-and-language pre-training (VLP) models have experienced a surge in
popularity recently. By fine-tuning them on specific datasets, significant
performance improvements have been observed in various tasks. However, full
fine-tuning of VLP models not only consumes a significant amount of
computational resources but also has a significant environmental impact.
Moreover, as remote sensing (RS) data is constantly being updated, full
fine-tuning may not be practical for real-world applications. To address this
issue, in this work, we investigate the parameter-efficient transfer learning
(PETL) method to effectively and efficiently transfer visual-language knowledge
from the natural domain to the RS domain on the image-text retrieval task. To
this end, we make the following contributions. 1) We construct a novel and
sophisticated PETL framework for the RS image-text retrieval (RSITR) task,
which includes the pretrained CLIP model, a multimodal remote sensing adapter,
and a hybrid multi-modal contrastive (HMMC) learning objective; 2) To deal with
the problem of high intra-modal similarity in RS data, we design a simple yet
effective HMMC loss; 3) We provide comprehensive empirical studies for
PETL-based RS image-text retrieval. Our results demonstrate that the proposed
method is promising and of great potential for practical applications. 4) We
benchmark extensive state-of-the-art PETL methods on the RSITR task. Our
proposed model only contains 0.16M training parameters, which can achieve a
parameter reduction of 98.9% compared to full fine-tuning, resulting in
substantial savings in training costs. Our retrieval performance exceeds
traditional methods by 7-13% and achieves comparable or better performance than
full fine-tuning. This work can provide new ideas and useful insights for RS
vision-language tasks.

通过参数高效的迁移学习方法，将自然领域的视觉 - 语言知识有效且高效地转移到遥感领域的图像 - 文本检索任务上，提出了一个新颖而复杂的参数高效转移学习框架，通过预训练的 CLIP 模型、多模态遥感适配器和混合多模态对比学习目标来解决遥感数据中高内部相似性的问题，并在 RSITR 任务上进行了全面的实证研究，结果表明该方法具有很大的应用潜力，可以在实际应用中实现大幅度的训练成本节省，超过传统方法 7-13% 的检索性能并达到与全面微调相媲美甚至更好的性能，为 RS 视觉 - 语言任务提供了新的思路和有用的见解。