Source-Free Domain Adaptation (SFDA) aims to adapt a source model for a
target domain, with only access to unlabeled target training data and the
source model pre-trained on a supervised source domain. Relying on pseudo
labeling and/or auxiliary supervision, conventional methods are inevitably
error-prone. To mitigate this limitation, in this work we for the first time
explore the potentials of off-the-shelf vision-language (ViL) multimodal models
(e.g.,CLIP) with rich whilst heterogeneous knowledge. We find that directly
applying the ViL model to the target domain in a zero-shot fashion is
unsatisfactory, as it is not specialized for this particular task but largely
generic. To make it task specific, we propose a novel Distilling multimodal
Foundation model(DIFO)approach. Specifically, DIFO alternates between two steps
during adaptation: (i) Customizing the ViL model by maximizing the mutual
information with the target model in a prompt learning manner, (ii) Distilling
the knowledge of this customized ViL model to the target model. For more
fine-grained and reliable distillation, we further introduce two effective
regularization terms, namely most-likely category encouragement and predictive
consistency. Extensive experiments show that DIFO significantly outperforms the
state-of-the-art alternatives. Our source code will be released.

源无关领域适应 (SFDA) 旨在适应一个源模型到目标领域，只能访问标记有监督源领域的预先训练的模型和无标记目标培训数据。我们首次探索了富含异构知识的现成视觉 - 语言多模态模型 (例如 CLIP) 的潜力，提出了一种新颖的蒸馏多模态基础模型 (DIFO) 方法来解决现有方法中不可避免的错误问题，并通过两个步骤的交替来使其任务特定化，实验结果表明 DIFO 显著优于现有的替代方法。