In this paper, we present the Semantic Boundary Conditioned Backbone (SBCB)
framework, a simple yet effective training framework that is model-agnostic and
boosts segmentation performance, especially around the boundaries. Motivated by
the recent development in improving semantic segmentation by incorporating
boundaries as auxiliary tasks, we propose a multi-task framework that uses
semantic boundary detection (SBD) as an auxiliary task. The SBCB framework
utilizes the nature of the SBD task, which is complementary to semantic
segmentation, to improve the backbone of the segmentation head. We apply an SBD
head that exploits the multi-scale features from the backbone, where the model
learns low-level features in the earlier stages, and high-level semantic
understanding in the later stages. This head perfectly complements the common
semantic segmentation architectures where the features from the later stages
are used for classification. We can improve semantic segmentation models
without additional parameters during inference by only conditioning the
backbone. Through extensive evaluations, we show the effectiveness of the SBCB
framework by improving various popular segmentation heads and backbones by 0.5%
~ 3.0% IoU on the Cityscapes dataset and gains 1.6% ~ 4.1% in boundary Fscores.
We also apply this framework on customized backbones and the emerging vision
transformer models and show the effectiveness of the SBCB framework.

本文提出了 Semantic Boundary Conditioned Backbone (SBCB) 框架，该框架是一种简单而有效的训练框架，旨在提高分割性能，尤其是在边界周围，可用于改善各种流行分割头和骨干网络。

语义边界辅助的语义分割

Boosting Semantic Segmentation with Semantic Boundaries

This paper presents the first attempt to learn semantic boundary detection
using image-level class labels as supervision. Our method starts by estimating
coarse areas of object classes through attentions drawn by an image
classification network. Since boundaries will locate somewhere between such
areas of different classes, our task is formulated as a multiple instance
learning (MIL) problem, where pixels on a line segment connecting areas of two
different classes are regarded as a bag of boundary candidates. Moreover, we
design a new neural network architecture that can learn to estimate semantic
boundaries reliably even with uncertain supervision given by the MIL strategy.
Our network is used to generate pseudo semantic boundary labels of training
images, which are in turn used to train fully supervised models. The final
model trained with our pseudo labels achieves an outstanding performance on the
SBD dataset, where it is as competitive as some of previous arts trained with
stronger supervision.

本文介绍了第一次尝试使用图像级别的类别标签作为监督来学习语义边界检测的方法，并且提出了一种基于多实例学习的任务形式，在区别不同类别的领域之间通过互相连线代表边界。我们提出的神经网络模型可以可靠地学习估计语义边界，即使是在多实例学习策略的不确定监督下，将其用于生成训练图像的伪语义边界标签，并使用这些标签来训练完全监督模型。最终模型在 SBD 数据集上取得了出色的表现，与一些使用更强监督的模型相当竞争。