Pre-trained vision-language models (VLMs) have shown impressive performance
on various downstream tasks by utilizing knowledge learned from large data. In
general, the performance of VLMs on target tasks can be further improved by
prompt tuning, which adds context to the input image or text. By leveraging
data from target tasks, various prompt-tuning methods have been studied in the
literature. A key to prompt tuning is the feature space alignment between two
modalities via learnable vectors with model parameters fixed. We observed that
the alignment becomes more effective when embeddings of each modality are
`well-arranged' in the latent space. Inspired by this observation, we proposed
distribution-aware prompt tuning (DAPT) for vision-language models, which is
simple yet effective. Specifically, the prompts are learned by maximizing
inter-dispersion, the distance between classes, as well as minimizing the
intra-dispersion measured by the distance between embeddings from the same
class. Our extensive experiments on 11 benchmark datasets demonstrate that our
method significantly improves generalizability. The code is available at
this https URL

通过学习大规模数据的知识，预训练的视觉 - 语言模型（VLMs）在各种下游任务中表现出色。本文提出了分布感知的提示微调（DAPT），通过对齐两种模态之间的特征空间，并对每种模态的嵌入进行良好排列，显著改善了模型的泛化能力。

面向视觉 - 语言模型的分布感知提示调整

Distribution-Aware Prompt Tuning for Vision-Language Models

Heterogeneous unsupervised domain adaptation (HUDA) is the most challenging
domain adaptation setting where the feature space differs between source and
target domains, and the target domain has only unlabeled data. Existing HUDA
methods assume that both positive and negative examples are available in the
source domain, which may not be satisfied in some real applications. This paper
addresses a new challenging setting called positive and unlabeled heterogeneous
domain adaptation (PU-HDA), a HUDA setting where the source domain only has
positives. PU-HDA can also be viewed as an extension of PU learning where the
positive and unlabeled examples are sampled from different domains. A naive
combination of existing HUDA and PU learning methods is ineffective in PU-HDA
due to the gap in label distribution between the source and target domains. To
overcome this issue, we propose a novel method, positive-adversarial domain
adaptation (PADA), which can predict likely positive examples from the
unlabeled target data and simultaneously align the feature spaces to reduce the
distribution divergence between the whole source data and the likely positive
target data. PADA achieves this by a unified adversarial training framework for
learning a classifier to predict positive examples and a feature transformer to
transform the target feature space to that of the source. Specifically, they
are both trained to fool a common discriminator that determines whether the
likely positive examples are from the target or source domain. We
experimentally show that PADA outperforms several baseline methods, such as the
naive combination of HUDA and PU learning.

该论文提出了一个新的异构无监督领域自适应 (UDA) 任务：正例与未标记数据异构领域自适应 (positive and unlabeled heterogeneous domain adaptation, PU-HDA)，并针对该任务提出了一种名为 PADA 的新颖方法，该方法通过对抗训练框架学习分类器，以预测可能的正例，并学习特征转换器将目标领域的特征空间对齐到源领域的特征空间，从而减小源域数据和目标域的可能正例数据之间的分布差异。