Unsupervised domain adaptation (UDA) aims to learn a well-performed model in an unlabeled target domain by leveraging labeled data from one or multiple related source domains. It remains a great challenge due to 1) the lack of annotations in the target domain and 2) the rich discrepancy between the distributions of source and target data. We propose Spectral UDA (SUDA), an efficient yet effective UDA technique that works in the spectral space and is generic across different visual recognition tasks in detection, classification and segmentation. SUDA addresses UDA challenges from two perspectives. First, it mitigates inter-domain discrepancies by a spectrum transformer (ST) that maps source and target images into spectral space and learns to enhance domain-invariant spectra while suppressing domain-variant spectra simultaneously. To this end, we design novel adversarial multi-head spectrum attention that leverages contextual information to identify domain-variant and domain-invariant spectra effectively. Second, it mitigates the lack of annotations in target domain by introducing multi-view spectral learning which aims to learn comprehensive yet confident target representations by maximizing the mutual information among multiple ST augmentations capturing different spectral views of each target sample. Extensive experiments over different visual tasks (e.g., detection, classification and segmentation) show that SUDA achieves superior accuracy and it is also complementary with state-of-the-art UDA methods with consistent performance boosts but little extra computation.

本文提出了一种可以在不同的视觉识别任务中推广的有效和高效的无监督域自适应技术Spectral UDA（SUDA），该技术从两个方面解决了UDA的挑战，第一，引入了谱变换器（ST），通过同时增强源样本和目标样本的域不变谱的同时抑制域变谱来缓解跨域不一致性。第二，引入了多视域谱学习，通过最大化目标样本的多个ST生成的谱视图之间的互信息来学习有用的无监督特征表示。经过广泛的实验，SUDA在目标检测、语义分割和图像分类等不同视觉任务中始终表现出优秀的准确性。此外，SUDA还可以与基于Transformer的网络结合使用，在目标检测方面取得了最先进的成果。

视觉识别的光谱无监督域自适应