In this work we introduce a novel, CNN-based architecture that can be trained
end-to-end to deliver seamless scene segmentation results. Our goal is to
predict consistent semantic segmentation and detection results by means of a
panoptic output format, going beyond the simple combination of independently
trained segmentation and detection models. The proposed architecture takes
advantage of a novel segmentation head that seamlessly integrates multi-scale
features generated by a Feature Pyramid Network with contextual information
conveyed by a light-weight DeepLab-like module. As additional contribution we
review the panoptic metric and propose an alternative that overcomes its
limitations when evaluating non-instance categories. Our proposed network
architecture yields state-of-the-art results on three challenging street-level
datasets, i.e. Cityscapes, Indian Driving Dataset and Mapillary Vistas.

本研究提出了一种基于 CNN 的新型架构，可以进行端到端的训练，以提供无缝的场景分割结果。该网络结构采用了一种新颖的分割头，将由特征金字塔网络生成的多尺度特征与由轻量级 DeepLab 模块传递的上下文信息进行了无缝整合，从而预测一致的语义分割和检测结果。此外，研究人员还提出了一种替代 panoptic 度量的方法，克服了评估非实例类别时的局限性。通过在三个具有挑战性的街景数据集上测试，即 Cityscapes、Indian Driving Dataset 和 Mapillary Vistas，我们的网络架构取得了最先进的结果。