In this paper, we first assess and harness various Vision Foundation Models
(VFMs) in the context of Domain Generalized Semantic Segmentation (DGSS).
Driven by the motivation that Leveraging Stronger pre-trained models and Fewer
trainable parameters for Superior generalizability, we introduce a robust
fine-tuning approach, namely Rein, to parameter-efficiently harness VFMs for
DGSS. Built upon a set of trainable tokens, each linked to distinct instances,
Rein precisely refines and forwards the feature maps from each layer to the
next layer within the backbone. This process produces diverse refinements for
different categories within a single image. With fewer trainable parameters,
Rein efficiently fine-tunes VFMs for DGSS tasks, surprisingly surpassing full
parameter fine-tuning. Extensive experiments across various settings
demonstrate that Rein significantly outperforms state-of-the-art methods.
Remarkably, with just an extra 1% of trainable parameters within the frozen
backbone, Rein achieves a mIoU of 68.1% on the Cityscapes, without accessing
any real urban-scene datasets.

该研究中，我们首先评估和利用各种视觉基础模型（Vision Foundation Models）在域泛化语义分割（Domain Generalized Semantic Segmentation）中的应用。我们引入了一种名为 Rein 的强大微调方法，通过利用更强的预训练模型和更少的可训练参数来提高模型的泛化能力。实验证明，Rein 在各种设置下明显优于现有方法，并在 Cityscapes 数据集上获得了令人惊讶的 68.1% 的 mIoU，而仅使用了额外 1% 的可训练参数。