Semantic segmentation is a fundamental task in visual scene understanding. We
focus on the supervised setting, where ground-truth semantic annotations are
available. Based on knowledge about the high regularity of real-world scenes,
we propose a method for improving class predictions by learning to selectively
exploit information from neighboring pixels. In particular, our method is based
on the prior that for each pixel, there is a seed pixel in its close
neighborhood sharing the same prediction with the former. Motivated by this
prior, we design a novel two-head network, named Offset Vector Network
(OVeNet), which generates both standard semantic predictions and a dense 2D
offset vector field indicating the offset from each pixel to the respective
seed pixel, which is used to compute an alternative, seed-based semantic
prediction. The two predictions are adaptively fused at each pixel using a
learnt dense confidence map for the predicted offset vector field. We supervise
offset vectors indirectly via optimizing the seed-based prediction and via a
novel loss on the confidence map. Compared to the baseline state-of-the-art
architectures HRNet and HRNet+OCR on which OVeNet is built, the latter achieves
significant performance gains on two prominent benchmarks for semantic
segmentation of driving scenes, namely Cityscapes and ACDC. Code is available
at this https URL

通过 Offset Vector Network 将近邻像素信息应用到类别预测中，从而实现图像语义分割的改进， 提出了一种新的两头网络，该网络生成标准的语义预测和密集的 2D 偏移向量场，指示每个像素到相应种子像素的偏移，并通过学习密集置信度图进行自适应融合。在 Cityscapes 和 ACDC 数据集上，与 HRNet 和 HRNet + OCR 相比，OVE-Net 取得了显着的性能提升。