Semantic segmentation models are limited in their ability to scale to large
numbers of object classes. In this paper, we introduce the new task of
zero-shot semantic segmentation: learning pixel-wise classifiers for never-seen
object categories with zero training examples. To this end, we present a novel
architecture, ZS3Net, combining a deep visual segmentation model with an
approach to generate visual representations from semantic word embeddings. By
this way, ZS3Net addresses pixel classification tasks where both seen and
unseen categories are faced at test time (so called "generalized" zero-shot
classification). Performance is further improved by a self-training step that
relies on automatic pseudo-labeling of pixels from unseen classes. On the two
standard segmentation datasets, Pascal-VOC and Pascal-Context, we propose
zero-shot benchmarks and set competitive baselines. For complex scenes as ones
in the Pascal-Context dataset, we extend our approach by using a graph-context
encoding to fully leverage spatial context priors coming from class-wise
segmentation maps.

本文提出了一个新的任务：零样本语义分割，要求学习像素级分类器从未见过的物体类别中进行分类。ZS3Net 是本文提出的一种新架构，它将深度视觉分割模型与从语义词嵌入生成视觉表示的方法相结合，以实现在测试时间处理已见和未见类别的像素分类任务。本文还通过自我训练进一步提高了性能，并在 Pascal-VOC 和 Pascal-Context 两个标准分割数据集上提出了零样本基准并设立竞争基线。此外，对于像 Pascal-Context 数据集中的复杂场景，我们通过使用图形上下文编码来完全利用来自类别分割地图的空间上下文先验知识。