One fundamental challenge in building an instance segmentation model for a large number of classes in complex scenes is the lack of training examples, especially for rare objects. In this paper, we explore the possibility to increase the training examples without laborious data collection and annotation. We find that an abundance of instance segments can potentially be obtained freely from object-centric im-ages, according to two insights: (i) an object-centric image usually contains one salient object in a simple background; (ii) objects from the same class often share similar appearances or similar contrasts to the background. Motivated by these insights, we propose a simple and scalable framework FreeSeg for extracting and leveraging these "free" object foreground segments to facilitate model training in long-tailed instance segmentation. Concretely, we employ off-the-shelf object foreground extraction techniques (e.g., image co-segmentation) to generate instance mask candidates, followed by segments refinement and ranking. The resulting high-quality object segments can be used to augment the existing long-tailed dataset, e.g., by copying and pasting the segments onto the original training images. On the LVIS benchmark, we show that FreeSeg yields substantial improvements on top of strong baselines and achieves state-of-the-art accuracy for segmenting rare object categories.

本研究提出一个简单和可扩展的 FreeSeg 框架，可以利用物体中心的图像提取和利用物体前景中的“免费”多样性来促进复杂场景中类别繁多的实例分割模型的培训，通过对同一类别的物体中心形象的相似性进行调查，提出前景实例的候选片段，然后通过对片段质量的新排序获得高质量的物体片段，从而改善了弱数据问题并取得了最佳效果。

基于自由对象分割的长尾实例分割学习