This paper explores zero-label learning in Natural Language Processing (NLP), whereby no human-annotated data is used anywhere during training and models are trained purely on synthetic data. At the core of our framework is a novel approach for better leveraging the powerful pretrained language models. Specifically, inspired by the recent success of few-shot inference on GPT-3, we present a training data creation procedure named Unsupervised Data Generation (UDG), which leverages few-shot prompts to synthesize high-quality training data without real human annotations. Our method enables zero-label learning as we train task-specific models solely on the synthetic data, yet we achieve better or comparable results from strong baseline models trained on human-labeled data. Furthermore, when mixed with labeled data, our approach serves as a highly effective data augmentation procedure, achieving new state-of-the-art results on the SuperGLUE benchmark.

该文章探讨了零标签学习在自然语言处理中的应用，介绍了基于预训练语言模型的一种新方法Unsupervised Data Generation，能够生成高质量的训练数据，无需人工标注。该方法使得用不带标签的数据训练具体任务的模型成为可能，并且当与带标签的数据混合使用时，能实现高效的数据增强，并达到了SuperGLUE基准测试的新最优结果。

零标签语言学习的探索