Conventional approaches to text classification typically assume the existence
of a fixed set of predefined labels to which a given text can be classified.
However, in real-world applications, there exists an infinite label space for
describing a given text. In addition, depending on the aspect (sentiment,
topic, etc.) and domain of the text (finance, legal, etc.), the interpretation
of the label can vary greatly. This makes the task of text classification,
particularly in the zero-shot scenario, extremely challenging. In this paper,
we investigate the task of zero-shot text classification with the aim of
improving the ability of pre-trained language models (PLMs) to generalize to
both seen and unseen data across varying aspects and domains. To solve this we
introduce two new simple yet effective pre-training strategies, Implicit and
Explicit pre-training. These methods inject aspect-level understanding into the
model at train time with the goal of conditioning the model to build task-level
understanding. To evaluate this, we construct and release UTCD, a new benchmark
dataset for evaluating text classification in zero-shot settings. Experimental
results on UTCD show that our approach achieves improved zero-shot
generalization on a suite of challenging datasets across an array of zero-shot
formalizations.

本文旨在探究利用两种新的有效的预训练策略（Implicit 和 Explicit pre-training）来改善 PLMs 在不同方面与领域的数据上泛化能力，特别是在零样本情况下进行文本分类的任务。作者通过引入 UTCD 数据集进行评估，证明了该方法在一系列零样本范式的挑战数据集上实现了更好的泛化能力。