In this paper, we introduce a realistic and challenging domain adaptation
problem called Universal Semi-supervised Model Adaptation (USMA), which i)
requires only a pre-trained source model, ii) allows the source and target
domain to have different label sets, i.e., they share a common label set and
hold their own private label set, and iii) requires only a few labeled samples
in each class of the target domain. To address USMA, we propose a collaborative
consistency training framework that regularizes the prediction consistency
between two models, i.e., a pre-trained source model and its variant
pre-trained with target data only, and combines their complementary strengths
to learn a more powerful model. The rationale of our framework stems from the
observation that the source model performs better on common categories than the
target-only model, while on target-private categories, the target-only model
performs better. We also propose a two-perspective, i.e., sample-wise and
class-wise, consistency regularization to improve the training. Experimental
results demonstrate the effectiveness of our method on several benchmark
datasets.

提出了一种称为 Universal Semi-supervised Model Adaptation (USMA) 的现实和具有挑战性的领域适应问题，该问题需要预训练的源模型，允许源域和目标域具有不同的标签集，并且在目标域中每个类别只需要少量标记样本。为了解决 USMA 问题，提出了一种协同一致性训练框架，通过在两个模型之间正则化预测一致性来结合它们的互补优势，以学习更强大的模型，并提出了一种基于样本和类别的一致性正则化方法来改善训练过程。在几个基准数据集上的实验证明了我们方法的有效性。

通过协同一致性训练实现通用半监督模型适应

Universal Semi-supervised Model Adaptation via Collaborative Consistency  Training

Large language models have improved zero-shot text classification by allowing
the transfer of semantic knowledge from the training data in order to classify
among specific label sets in downstream tasks. We propose a simple way to
further improve zero-shot accuracies with minimal effort. We curate small
finetuning datasets intended to describe the labels for a task. Unlike typical
finetuning data, which has texts annotated with labels, our data simply
describes the labels in language, e.g., using a few related terms,
dictionary/encyclopedia entries, and short templates. Across a range of topic
and sentiment datasets, our method is more accurate than zero-shot by 15-17%
absolute. It is also more robust to choices required for zero-shot
classification, such as patterns for prompting the model to classify and
mappings from labels to tokens in the model's vocabulary. Furthermore, since
our data merely describes the labels but does not use input texts, finetuning
on it yields a model that performs strongly on multiple text domains for a
given label set, even improving over few-shot out-of-domain classification in
multiple settings.

本研究提出了一种简单的方法来进一步提高零 - shot 分类准确性，即利用精心策划的微调数据集描述任务的标签，进而在多个文本域中获得强大的预测模型。