Recent work has demonstrated that pre-trained language models (PLMs) are
zero-shot learners. However, most existing zero-shot methods involve heavy
human engineering or complicated self-training pipelines, hindering their
application to new situations. In this work, we show that zero-shot text
classification can be improved simply by clustering texts in the embedding
spaces of PLMs. Specifically, we fit the unlabeled texts with a Bayesian
Gaussian Mixture Model after initializing cluster positions and shapes using
class names. Despite its simplicity, this approach achieves superior or
comparable performance on both topic and sentiment classification datasets and
outperforms prior works significantly on unbalanced datasets. We further
explore the applicability of our clustering approach by evaluating it on 14
datasets with more diverse topics, text lengths, and numbers of classes. Our
approach achieves an average of 20% absolute improvement over prompt-based
zero-shot learning. Finally, we compare different PLM embedding spaces and find
that texts are well-clustered by topics even if the PLM is not explicitly
pre-trained to generate meaningful sentence embeddings. This work indicates
that PLM embeddings can categorize texts without task-specific fine-tuning,
thus providing a new way to analyze and utilize their knowledge and zero-shot
learning ability.

本文通过将聚类方法应用于提前训练的语言模型的嵌入空间中，展示了在主题和情感分类数据集上，该方法在无需另作准备的情况下显著提高了零 - shot 文本分类的性能；并发现该方法不需要 Fine-tuning 就可以将文本分类；最后，文章还比较了不同 PLM 嵌入空间，并发现即使 PLM 未明确预训练为生成有意义的句子嵌入，它仍然能按主题将文本很好地聚类。

通过聚类表示，使预训练语言模型成为更好的零样本学习器

Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Non-exhaustive learning (NEL) is an emerging machine-learning paradigm
designed to confront the challenge of non-stationary environments characterized
by anon-exhaustive training sets lacking full information about the available
classes.Unlike traditional supervised learning that relies on fixed models, NEL
utilizes self-adjusting machine learning to better accommodate the
non-stationary nature of the real-world problem, which is at the root of many
recently discovered limitations of deep learning. Some of these hurdles led to
a surge of interest in several research areas relevant to NEL such as open set
classification or zero-shot learning. The presented study which has been
motivated by two important applications proposes a NEL algorithm built on a
highly flexible, doubly non-parametric Bayesian Gaussian mixture model that can
grow arbitrarily large in terms of the number of classes and their components.
We report several experiments that demonstrate the promising performance of the
introduced model for NEL.

这篇论文介绍了一种基于双非参数贝叶斯高斯混合模型的非穷举学习算法，它能够适应真实问题中的非平稳性质，并可应用于包含无穷多种类及其组件的问题。多次实验表明，该模型在非穷举学习中取得了良好的性能。