Domain adaptation has been extensively investigated in computer vision but still requires access to target data at the training time, which might be difficult to obtain in some uncommon conditions. In this paper, we present a new framework for domain adaptation relying on a single Vision-Language (VL) latent embedding instead of full target data. First, leveraging a contrastive language-image pre-training model (CLIP), we propose prompt/photo-driven instance normalization (PIN). PIN is a feature augmentation method that mines multiple visual styles using a single target VL latent embedding, by optimizing affine transformations of low-level source features. The VL embedding can come from a language prompt describing the target domain, a partially optimized language prompt, or a single unlabeled target image. Second, we show that these mined styles (i.e., augmentations) can be used for zero-shot (i.e., target-free) and one-shot unsupervised domain adaptation. Experiments on semantic segmentation demonstrate the effectiveness of the proposed method, which outperforms relevant baselines in the zero-shot and one-shot settings.

本研究针对领域适应过程中的目标数据获取困难问题，提出了一种新的基于单一视觉-语言嵌入的方法。该方法通过对低级源特征的仿射变换进行优化，实现了一种特征增强方法，从而有效利用了多种视觉风格进行零样本和单样本无监督领域适应。实验表明，所提方法在语义分割任务中超过了相关基线，展现了其有效性。

基于单一视觉-语言嵌入的领域适应