Panoptic segmentation that unifies instance segmentation and semantic
segmentation has recently attracted increasing attention. While most existing
methods focus on designing novel architectures, we steer toward a different
perspective: performing automated multi-loss adaptation (named Ada-Segment) on
the fly to flexibly adjust multiple training losses over the course of training
using a controller trained to capture the learning dynamics. This offers a few
advantages: it bypasses manual tuning of the sensitive loss combination, a
decisive factor for panoptic segmentation; it allows to explicitly model the
learning dynamics, and reconcile the learning of multiple objectives (up to ten
in our experiments); with an end-to-end architecture, it generalizes to
different datasets without the need of re-tuning hyperparameters or
re-adjusting the training process laboriously. Our Ada-Segment brings 2.7%
panoptic quality (PQ) improvement on COCO val split from the vanilla baseline,
achieving the state-of-the-art 48.5% PQ on COCO test-dev split and 32.9% PQ on
ADE20K dataset. The extensive ablation studies reveal the ever-changing
dynamics throughout the training process, necessitating the incorporation of an
automated and adaptive learning strategy as presented in this paper.

本文提出自适应分段（Ada-Segment）方法，通过自适应地调节多个培训损失，以涵盖多个目标，并以一个端到端的架构来概括性地处理分段任务，从而实现了对各种数据集的可靠泛化，成为目前的最优解。