Few-shot segmentation is a task to segment objects or regions of novel
classes within an image given only a few annotated examples. In the generalized
setting, the task extends to segment both the base and the novel classes. The
main challenge is how to train the model such that the addition of novel
classes does not hurt the base classes performance, also known as catastrophic
forgetting. To mitigate this issue, we use SegGPT as our base model and train
it on the base classes. Then, we use separate learnable prompts to handle
predictions for each novel class. To handle various object sizes which
typically present in remote sensing domain, we perform patch-based prediction.
To address the discontinuities along patch boundaries, we propose a
patch-and-stitch technique by re-framing the problem as an image inpainting
task. During inference, we also utilize image similarity search over image
embeddings for prompt selection and novel class filtering to reduce false
positive predictions. Based on our experiments, our proposed method boosts the
weighted mIoU of a simple fine-tuned SegGPT from 15.96 to 35.08 on the
validation set of few-shot OpenEarthMap dataset given in the challenge.

通过使用 SegGPT 作为基础模型，分别为每个新类别使用可学习的提示语进行预测，并通过图像修复任务解决遥感领域中存在的对象大小不一致性和补丁边界的不连续性问题，同时利用图像嵌入的相似性搜索来选择提示语和减少误报预测，实验结果表明我们的方法将简单微调的 SegGPT 在 Few-shot OpenEarthMap 数据集验证集上的加权 mIoU 从 15.96 提升到 35.08。

遥感领域少样本语义分割的可学习提示

Learnable Prompt for Few-Shot Semantic Segmentation in Remote Sensing  Domain

Co-salient object detection targets at detecting co-existed salient objects
among a group of images. Recently, a generalist model for segmenting everything
in context, called SegGPT, is gaining public attention. In view of its
breakthrough for segmentation, we can hardly wait to probe into its
contribution to the task of co-salient object detection. In this report, we
first design a framework to enable SegGPT for the problem of co-salient object
detection. Proceed to the next step, we evaluate the performance of SegGPT on
the problem of co-salient object detection on three available datasets. We
achieve a finding that co-saliency scenes challenges SegGPT due to context
discrepancy within a group of co-saliency images.

本文将 SegGPT 应用于共显目标检测，结果发现其在面临组内上下文差异时的性能存在挑战。