Self-training based on pseudo-labels has emerged as a dominant approach for
addressing conditional distribution shifts in unsupervised domain adaptation
(UDA) for semantic segmentation problems. A notable drawback, however, is that
this family of approaches is susceptible to erroneous pseudo labels that arise
from confirmation biases in the source domain and that manifest as nuisance
factors in the target domain. A possible source for this mismatch is the
reliance on only photometric cues provided by RGB image inputs, which may
ultimately lead to sub-optimal adaptation. To mitigate the effect of mismatched
pseudo-labels, we propose to incorporate structural cues from auxiliary
modalities, such as depth, to regularise conventional self-training objectives.
Specifically, we introduce a contrastive pixel-level objectness constraint that
pulls the pixel representations within a region of an object instance closer,
while pushing those from different object categories apart. To obtain object
regions consistent with the true underlying object, we extract information from
both depth maps and RGB-images in the form of multimodal clustering. Crucially,
the objectness constraint is agnostic to the ground-truth semantic labels and,
hence, appropriate for unsupervised domain adaptation. In this work, we show
that our regularizer significantly improves top performing self-training
methods (by up to $2$ points) in various UDA benchmarks for semantic
segmentation. We include all code in the supplementary.

该研究提出一种基于深度信息的结构性正则化方法，将物体对比约束融入传统的自学习目标中，通过 RGB 图像和深度图像的多模态聚类，实现对真实物体的一致性提取。在多个无监督领域自适应测试中，改进的方法在语义分割方面取得了显著效果提升。