As a novel and effective fine-tuning paradigm based on large-scale
pre-trained language models (PLMs), prompt-tuning aims to reduce the gap
between downstream tasks and pre-training objectives. While prompt-tuning has
yielded continuous advancements in various tasks, such an approach still
remains a persistent defect: prompt-tuning methods fail to generalize to
specific few-shot patterns. From the perspective of distribution analyses, we
disclose that the intrinsic issues behind the phenomenon are the
over-multitudinous conceptual knowledge contained in PLMs and the abridged
knowledge for target downstream domains, which jointly result in that PLMs
mis-locate the knowledge distributions corresponding to the target domains in
the universal knowledge embedding space. To this end, we intuitively explore to
approximate the unabridged target domains of downstream tasks in a debiased
manner, and then abstract such domains to generate discriminative prompts,
thereby providing the de-ambiguous guidance for PLMs. Guided by such an
intuition, we propose a simple yet effective approach, namely BayesPrompt, to
learn prompts that contain the domain discriminative information against the
interference from domain-irrelevant knowledge. BayesPrompt primitively
leverages known distributions to approximate the debiased factual distributions
of target domains and further uniformly samples certain representative features
from the approximated distributions to generate the ultimate prompts for PLMs.
We provide theoretical insights with the connection to domain adaptation.
Empirically, our method achieves state-of-the-art performance on benchmarks.

通过将已知分布近似为去偏的目标领域的真实分布，并从近似分布中均匀采样一定的代表性特征，生成 PLMs 的最终提示，我们的方法在基准测试中实现了最先进的性能。