In recent years, transformer-based models have dominated panoptic
segmentation, thanks to their strong modeling capabilities and their unified
representation for both semantic and instance classes as global binary masks.
In this paper, we revisit pure convolution model and propose a novel panoptic
architecture named MaskConver. MaskConver proposes to fully unify things and
stuff representation by predicting their centers. To that extent, it creates a
lightweight class embedding module that can break the ties when multiple
centers co-exist in the same location. Furthermore, our study shows that the
decoder design is critical in ensuring that the model has sufficient context
for accurate detection and segmentation. We introduce a powerful ConvNeXt-UNet
decoder that closes the performance gap between convolution- and
transformerbased models. With ResNet50 backbone, our MaskConver achieves 53.6%
PQ on the COCO panoptic val set, outperforming the modern convolution-based
model, Panoptic FCN, by 9.3% as well as transformer-based models such as
Mask2Former (+1.7% PQ) and kMaX-DeepLab (+0.6% PQ). Additionally, MaskConver
with a MobileNet backbone reaches 37.2% PQ, improving over Panoptic-DeepLab by
+6.4% under the same FLOPs/latency constraints. A further optimized version of
MaskConver achieves 29.7% PQ, while running in real-time on mobile devices. The
code and model weights will be publicly available

该研究论文提出了一种名为 MaskConver 的新型全景分割架构，通过预测物体和背景的中心来实现事物和物质的完全统一表示。通过引入 ConvNeXt-UNet 解码器，该模型在 COOC 全景验证集上相比基于卷积和 transformer 的模型以及现代基于卷积的模型和 transformer-based models 取得了更好的性能，此外，优化后的 MaskConver 在移动设备上实时运行，并在相同 FLOPs / 延迟限制下提高了 6.4% 的 PQ 值。