The goal of this paper is to perform object detection in satellite imagery
with only a few examples, thus enabling users to specify any object class with
minimal annotation. To this end, we explore recent methods and ideas from
open-vocabulary detection for the remote sensing domain. We develop a few-shot
object detector based on a traditional two-stage architecture, where the
classification block is replaced by a prototype-based classifier. A large-scale
pre-trained model is used to build class-reference embeddings or prototypes,
which are compared to region proposal contents for label prediction. In
addition, we propose to fine-tune prototypes on available training images to
boost performance and learn differences between similar classes, such as
aircraft types. We perform extensive evaluations on two remote sensing datasets
containing challenging and rare objects. Moreover, we study the performance of
both visual and image-text features, namely DINOv2 and CLIP, including two CLIP
models specifically tailored for remote sensing applications. Results indicate
that visual features are largely superior to vision-language models, as the
latter lack the necessary domain-specific vocabulary. Lastly, the developed
detector outperforms fully supervised and few-shot methods evaluated on the
SIMD and DIOR datasets, despite minimal training parameters.

本文旨在使用仅有少量示例来进行卫星图像中的目标检测，从而使用户能够使用最少的注释来指定任何目标类。我们探讨了来自开放词汇检测和遥感领域的最新方法和思路。我们基于传统的两阶段架构开发了一个少样本目标检测器，其中分类块被基于原型的分类器替代。我们使用大规模预训练模型构建类参考嵌入或原型，并与区域建议内容进行标签预测。另外，我们提出在可用的训练图像上微调原型，以提高性能并学习类似类别之间的差异，例如飞机类型。我们对包含具有挑战性和罕见对象的两个遥感数据集进行了广泛评估。此外，我们研究了视觉和图像文本特征的性能，即 DINOv2 和 CLIP，包括专门针对遥感应用的两个 CLIP 模型。结果表明，视觉特征在很大程度上优于视觉语言模型，因为后者缺乏必要的领域特定词汇。最后，尽管训练参数很少，但开发的检测器在 SIMD 和 DIOR 数据集上表现出优于全监督和少样本方法的性能。