In this paper, we propose a unified panoptic segmentation network (UPSNet)
for tackling the newly proposed panoptic segmentation task. On top of a single
backbone residual network, we first design a deformable convolution based
semantic segmentation head and a Mask R-CNN style instance segmentation head
which solve these two subtasks simultaneously. More importantly, we introduce a
parameter-free panoptic head which solves the panoptic segmentation via
pixel-wise classification. It first leverages the logits from the previous two
heads and then innovatively expands the representation for enabling prediction
of an extra unknown class which helps better resolve the conflicts between
semantic and instance segmentation. Additionally, it handles the challenge
caused by the varying number of instances and permits back propagation to the
bottom modules in an end-to-end manner. Extensive experimental results on
Cityscapes, COCO and our internal dataset demonstrate that our UPSNet achieves
state-of-the-art performance with much faster inference. Code has been made
available at: this https URL

本文提出了一种统一的全景分割网络（UPSNet），通过在单个骨干残差网络上设计基于可变卷积的语义分割头和基于 Mask R-CNN 的实例分割头，同时解决这两个子任务，并引入了一个无需参数的全景头来通过像素级分类解决全景分割，可帮助更好地解决语义分割和实例分割之间的冲突。此外，具有处理不同实例数量挑战的功能，可以端到端地在底部模块中反向传播。实验结果表明，UPSNet 可以实现最先进的性能，并具有更快的推理速度。