Few-shot segmentation is a task to segment objects or regions of novel classes within an image given only a few annotated examples. In the generalized setting, the task extends to segment both the base and the novel classes. The main challenge is how to train the model such that the addition of novel classes does not hurt the base classes performance, also known as catastrophic forgetting. To mitigate this issue, we use SegGPT as our base model and train it on the base classes. Then, we use separate learnable prompts to handle predictions for each novel class. To handle various object sizes which typically present in remote sensing domain, we perform patch-based prediction. To address the discontinuities along patch boundaries, we propose a patch-and-stitch technique by re-framing the problem as an image inpainting task. During inference, we also utilize image similarity search over image embeddings for prompt selection and novel class filtering to reduce false positive predictions. Based on our experiments, our proposed method boosts the weighted mIoU of a simple fine-tuned SegGPT from 15.96 to 35.08 on the validation set of few-shot OpenEarthMap dataset given in the challenge.

通过使用SegGPT作为基础模型，分别为每个新类别使用可学习的提示语进行预测，并通过图像修复任务解决遥感领域中存在的对象大小不一致性和补丁边界的不连续性问题，同时利用图像嵌入的相似性搜索来选择提示语和减少误报预测，实验结果表明我们的方法将简单微调的SegGPT在Few-shot OpenEarthMap数据集验证集上的加权mIoU从15.96提升到35.08。

遥感领域少样本语义分割的可学习提示