Pre-trained vision-language models (VLMs), such as CLIP, demonstrate impressive zero-shot classification capabilities with free-form prompts and even show some generalization in specialized domains. However, their performance on satellite imagery is limited due to the underrepresentation of such data in their training sets, which predominantly consist of ground-level images. Existing prompting techniques for satellite imagery are often restricted to generic phrases like a satellite image of ..., limiting their effectiveness for zero-shot land-use and land-cover (LULC) mapping. To address these challenges, we introduce SenCLIP, which transfers CLIPs representation to Sentinel-2 imagery by leveraging a large dataset of Sentinel-2 images paired with geotagged ground-level photos from across Europe. We evaluate SenCLIP alongside other SOTA remote sensing VLMs on zero-shot LULC mapping tasks using the EuroSAT and BigEarthNet datasets with both aerial and ground-level prompting styles. Our approach, which aligns ground-level representations with satellite imagery, demonstrates significant improvements in classification accuracy across both prompt styles, opening new possibilities for applying free-form textual descriptions in zero-shot LULC mapping.

本研究解决了已有视觉语言模型（VLM）在卫星图像中表现不佳的问题，主要由于训练数据中缺乏卫星图像。通过引入SenCLIP，我们利用欧洲的Sentinel-2图像与地理标记的地面照片配对，从而改善了对土地利用和覆盖（LULC）映射的零-shot分类准确性，开辟了使用自由形式文本描述的新可能性。

SenCLIP：通过地面提示增强Sentinel-2的零-shot土地利用映射