Albeit with varying degrees of progress in the field of Semi-Supervised
Semantic Segmentation, most of its recent successes are involved in unwieldy
models and the lightweight solution is still not yet explored. We find that
existing knowledge distillation techniques pay more attention to pixel-level
concepts from labeled data, which fails to take more informative cues within
unlabeled data into account. Consequently, we offer the first attempt to
provide lightweight SSSS models via a novel multi-granularity distillation
(MGD) scheme, where multi-granularity is captured from three aspects: i)
complementary teacher structure; ii) labeled-unlabeled data cooperative
distillation; iii) hierarchical and multi-levels loss setting. Specifically,
MGD is formulated as a labeled-unlabeled data cooperative distillation scheme,
which helps to take full advantage of diverse data characteristics that are
essential in the semi-supervised setting. Image-level semantic-sensitive loss,
region-level content-aware loss, and pixel-level consistency loss are set up to
enrich hierarchical distillation abstraction via structurally complementary
teachers. Experimental results on PASCAL VOC2012 and Cityscapes reveal that MGD
can outperform the competitive approaches by a large margin under diverse
partition protocols. For example, the performance of ResNet-18 and MobileNet-v2
backbone is boosted by 11.5% and 4.6% respectively under 1/16 partition
protocol on Cityscapes. Although the FLOPs of the model backbone is compressed
by 3.4-5.3x (ResNet-18) and 38.7-59.6x (MobileNetv2), the model manages to
achieve satisfactory segmentation results.

提出一种基于多层级知识蒸馏技术的轻量级半监督语义分割模型，采用标注数据和未标注数据的协同蒸馏方案，结合像素级一致性约束、语义感知层次损失和内容感知区域损失等多层次损失策略，从多个角度对模型知识进行精炼，能够在消耗更少的运算资源和更小的模型体积下取得同类模型不可比的性能提升。