Panoptic segmentation is posed as a new popular test-bed for the
state-of-the-art holistic scene understanding methods with the requirement of
simultaneously segmenting both foreground things and background stuff. The
state-of-the-art panoptic segmentation network exhibits high structural
complexity in different network components, i.e. backbone, proposal-based
foreground branch, segmentation-based background branch, and feature fusion
module across branches, which heavily relies on expert knowledge and tedious
trials. In this work, we propose an efficient, cooperative and highly automated
framework to simultaneously search for all main components including backbone,
segmentation branches, and feature fusion module in a unified panoptic
segmentation pipeline based on the prevailing one-shot Network Architecture
Search (NAS) paradigm. Notably, we extend the common single-task NAS into the
multi-component scenario by taking the advantage of the newly proposed
intra-modular search space and problem-oriented inter-modular search space,
which helps us to obtain an optimal network architecture that not only performs
well in both instance segmentation and semantic segmentation tasks but also be
aware of the reciprocal relations between foreground things and background
stuff classes. To relieve the vast computation burden incurred by applying NAS
to complicated network architectures, we present a novel path-priority greedy
search policy to find a robust, transferrable architecture with significantly
reduced searching overhead. Our searched architecture, namely Auto-Panoptic,
achieves the new state-of-the-art on the challenging COCO and ADE20K
benchmarks. Moreover, extensive experiments are conducted to demonstrate the
effectiveness of path-priority policy and transferability of Auto-Panoptic
across different datasets. Codes and models are available at:
this https URL

本文提出了一种高效、协作且高度自动化的框架，基于现有的一次性网络架构搜索（NAS）范式，在统一的全貌分割管道中同时搜索用于骨干、分割分支和特征融合模块的所有主要组件，将单一任务 NAS 扩展到多组件场景中，并采用新提出的模块内搜索空间和问题导向的模块间搜索空间，以帮助我们获得最佳网络架构，该架构不仅在实例分割和语义分割任务中表现良好，而且了解前景和背景物体类之间的相互关系，并通过引入一种新的路径优先贪心搜索策略来减少搜索开销，我们找到了一个稳健、可转移性强的架构，名为 Auto-Panoptic，并在具有挑战性的多个基准测试集上实现了新的最先进的结果。