A major thread of unsupervised domain adaptation (UDA) methods uses unlabeled
data from both source and target domains to learn domain-invariant
representations for adaptation. However, these methods showcase certain
limitations, encouraging the use of self-supervised learning through continued
pre-training. The necessity of continued pre-training or learning
domain-invariant representations is still unclear in the prompt-based
classification framework, where an input example is modified by a template and
then fed into a language model (LM) to generate a label string. To examine this
new paradigm of UDA in the prompt-based setup, we propose a frustratingly easy
UDA method (FEUDA) that trains an autoregressive LM on both unlabeled and
labeled examples using two different instruction-tuning tasks. Specifically,
the first task trains the LM on unlabeled texts from both domains via masked
language modeling (MLM), and the other uses supervised instruction-tuning on
source-labeled data for classification. We conduct extensive experiments on 24
real-world domain pairs to show the effectiveness of our method over strong
domain-invariant learning methods. Our analysis sheds light on why masked
language modeling improves target-domain classification performance in
prompt-based UDA. We discover that MLM helps the model learn both semantic and
background knowledge of a domain, which are both beneficial for downstream
classification.

通过在无标签数据上进行句子掩码模型训练（MLM）和源标记数据上进行监督指导训练，采用自监督学习和提示模型术语分类方法，我们提出了一种叫做困难易化领域适应（FEUDA）的方法，通过训练一个自回归语言模型，从源和目标领域的标签和无标签示例中，来学习领域不变表征，以提高目标领域的分类性能。

FEUDA: 极其简便的基于提示的无监督领域自适应

FEUDA: Frustratingly Easy Prompt Based Unsupervised Domain Adaptation

Prompt-based classification adapts tasks to a cloze question format utilizing
the [MASK] token and the filled tokens are then mapped to labels through
pre-defined verbalizers. Recent studies have explored the use of verbalizer
embeddings to reduce labor in this process. However, all existing studies
require a tuning process for either the pre-trained models or additional
trainable embeddings. Meanwhile, the distance between high-dimensional
verbalizer embeddings should not be measured by Euclidean distance due to the
potential for non-linear manifolds in the representation space. In this study,
we propose a tuning-free manifold-based space re-embedding method called
Locally Linear Embedding with Intra-class Neighborhood Constraint (LLE-INC) for
verbalizer embeddings, which preserves local properties within the same class
as guidance for classification. Experimental results indicate that even without
tuning any parameters, our LLE-INC is on par with automated verbalizers with
parameter tuning. And with the parameter updating, our approach further
enhances prompt-based tuning by up to 3.2%. Furthermore, experiments with the
LLaMA-7B&13B indicate that LLE-INC is an efficient tuning-free classification
approach for the hyper-scale language models.

利用无需调参的基于流形的重新嵌入的方法（LLE-INC）改进了基于提示的分类，并且在参数更新后可进一步提高分类性能。对 LLaMA-7B&13B 的实验结果表明，LLE-INC 是一种高效的无需调参的分类方法，适用于超大规模语言模型。