Image segmentation is often ambiguous at the level of individual image
patches and requires contextual information to reach label consensus. In this
paper we introduce Segmenter, a transformer model for semantic segmentation. In
contrast to convolution-based methods, our approach allows to model global
context already at the first layer and throughout the network. We build on the
recent Vision Transformer (ViT) and extend it to semantic segmentation. To do
so, we rely on the output embeddings corresponding to image patches and obtain
class labels from these embeddings with a point-wise linear decoder or a mask
transformer decoder. We leverage models pre-trained for image classification
and show that we can fine-tune them on moderate sized datasets available for
semantic segmentation. The linear decoder allows to obtain excellent results
already, but the performance can be further improved by a mask transformer
generating class masks. We conduct an extensive ablation study to show the
impact of the different parameters, in particular the performance is better for
large models and small patch sizes. Segmenter attains excellent results for
semantic segmentation. It outperforms the state of the art on both ADE20K and
Pascal Context datasets and is competitive on Cityscapes.

本文提出了 Segmenter，一种用于语义分割的 Transformer 模型，利用一种点刀线性解码器或一个 mask transformer 解码器将图像分割为不同的类别，通过对模型参数的分析，表明 Segmenter 在 ADE20K 和 Pascal Context 数据集上具有良好的性能。