Universal Multi-source Domain Adaptation (UniMDA) transfers knowledge from
multiple labeled source domains to an unlabeled target domain under domain
shifts (different data distribution) and class shifts (unknown target classes).
Existing solutions focus on excavating image features to detect unknown
samples, ignoring abundant information contained in textual semantics. In this
paper, we propose an Adaptive Prompt learning with Negative textual semantics
and uncErtainty modeling method based on Contrastive Language-Image
Pre-training (APNE-CLIP) for UniMDA classification tasks. Concretely, we
utilize the CLIP with adaptive prompts to leverage textual information of class
semantics and domain representations, helping the model identify unknown
samples and address domain shifts. Additionally, we design a novel global
instance-level alignment objective by utilizing negative textual semantics to
achieve more precise image-text pair alignment. Furthermore, we propose an
energy-based uncertainty modeling strategy to enlarge the margin distance
between known and unknown samples. Extensive experiments demonstrate the
superiority of our proposed method.

利用自适应提示学习、负面文本语义和基于对比语言 - 图像预训练的不确定性建模方法，本文提出了一种适应性提示方法，用于 UniMDA 分类任务，通过使用自适应提示的 CLIP 来利用类语义和领域表示的文本信息，帮助模型识别未知样本并解决领域转换，并提出了利用负面文本语义实现更精确的图像 - 文本对齐的全局实例级别对齐目标，以及一种基于能源的不确定性建模策略来扩大已知和未知样本之间的边际距离。大量实验证明了我们提出的方法的优越性。