Masked Image Modeling (MIM) has emerged as a pivotal approach for developing
foundational visual models in the field of remote sensing (RS). However,
current RS datasets are limited in volume and diversity, which significantly
constrains the capacity of MIM methods to learn generalizable representations.
In this study, we introduce \textbf{RS-4M}, a large-scale dataset designed to
enable highly efficient MIM training on RS images. RS-4M comprises 4 million
optical images encompassing abundant and fine-grained RS visual tasks,
including object-level detection and pixel-level segmentation. Compared to
natural images, RS images often contain massive redundant background pixels,
which limits the training efficiency of the conventional MIM models. To address
this, we propose an efficient MIM method, termed \textbf{SelectiveMAE}, which
dynamically encodes and reconstructs a subset of patch tokens selected based on
their semantic richness. SelectiveMAE roots in a progressive semantic token
selection module, which evolves from reconstructing semantically analogical
tokens to encoding complementary semantic dependencies. This approach
transforms conventional MIM training into a progressive feature learning
process, enabling SelectiveMAE to efficiently learn robust representations of
RS images. Extensive experiments show that SelectiveMAE significantly boosts
training efficiency by 2.2-2.7 times and enhances the classification,
detection, and segmentation performance of the baseline MIM model.The dataset,
source code, and trained models will be released.

这篇研究介绍了 RS-4M，一个大规模的数据集，用于在遥感图像上进行高效的 Masked Image Modeling（MIM）训练。同时提出了一种名为 SelectiveMAE 的高效 MIM 方法，通过动态编码和重构一部分基于语义丰富性选择的补丁令牌，显著提升了 MIM 模型的训练效率和基线模型的分类、检测和分割性能。