A key challenge for the widespread application of learning-based models for
robotic perception is to significantly reduce the required amount of annotated
training data while achieving accurate predictions. This is essential not only
to decrease operating costs but also to speed up deployment time. In this work,
we address this challenge for PAnoptic SegmenTation with fEw Labels (PASTEL) by
exploiting the groundwork paved by visual foundation models. We leverage
descriptive image features from such a model to train two lightweight network
heads for semantic segmentation and object boundary detection, using very few
annotated training samples. We then merge their predictions via a novel fusion
module that yields panoptic maps based on normalized cut. To further enhance
the performance, we utilize self-training on unlabeled images selected by a
feature-driven similarity scheme. We underline the relevance of our approach by
employing PASTEL to important robot perception use cases from autonomous
driving and agricultural robotics. In extensive experiments, we demonstrate
that PASTEL significantly outperforms previous methods for label-efficient
segmentation even when using fewer annotations. The code of our work is
publicly available at this http URL

通过利用视觉基础模型的描述性图像特征，我们训练了两个轻量级网络头进行语义分割和物体边界检测，并通过一种新颖的融合模块将它们的预测合并，以实现基于标准化切割的全景地图，从而显著提高了 PASTEL 在较少注释样本的情况下进行标签高效分割的性能。