Pre-trained Vision-Language Foundation Models utilizing extensive image-text
paired data have demonstrated unprecedented image-text association
capabilities, achieving remarkable results across various downstream tasks. A
critical challenge is how to make use of existing large-scale pre-trained VLMs,
which are trained on common objects, to perform the domain-specific transfer
for accomplishing domain-related downstream tasks. In this paper, we propose a
new framework that includes the Domain Foundation Model (DFM), bridging the gap
between the General Foundation Model (GFM) and domain-specific downstream
tasks. Moreover, we present an image-text paired dataset in the field of remote
sensing (RS), RS5M, which has 5 million RS images with English descriptions.
The dataset is obtained from filtering publicly available image-text paired
datasets and captioning label-only RS datasets with pre-trained VLM. These
constitute the first large-scale RS image-text paired dataset. Additionally, we
tried several Parameter-Efficient Fine-Tuning methods on RS5M to implement the
DFM. Experimental results show that our proposed dataset are highly effective
for various tasks, improving upon the baseline by $8 \% \sim 16 \%$ in
zero-shot classification tasks, and obtaining good results in both
Vision-Language Retrieval and Semantic Localization tasks. Finally, we show
successful results of training the RS Stable Diffusion model using the RS5M,
uncovering more use cases of the dataset.

本文提出了一种新的框架，包括领域基础模型（DFM），该模型缩小了通用基础模型（GFM）和特定领域下游任务之间的差距，并通过预训练的 VLM 将筛选后的遥感图像与英文描述进行配对，构成了第一个大规模遥感图像 - 文本匹配数据集。在该数据集上的实验结果表明，使用上述数据集和提出的 DFM 方法的零样本分类和视觉语言检索任务的性能都较好，并且 RS Stable Diffusion 模型的训练也取得了成功的结果。