This paper explores zero-label learning in Natural Language Processing (NLP),
whereby no human-annotated data is used anywhere during training and models are
trained purely on synthetic data. At the core of our framework is a novel
approach for better leveraging the powerful pretrained language models.
Specifically, inspired by the recent success of few-shot inference on GPT-3, we
present a training data creation procedure named Unsupervised Data Generation
(UDG), which leverages few-shot prompts to synthesize high-quality training
data without real human annotations. Our method enables zero-label learning as
we train task-specific models solely on the synthetic data, yet we achieve
better or comparable results from strong baseline models trained on
human-labeled data. Furthermore, when mixed with labeled data, our approach
serves as a highly effective data augmentation procedure, achieving new
state-of-the-art results on the SuperGLUE benchmark.

该文章探讨了零标签学习在自然语言处理中的应用，介绍了基于预训练语言模型的一种新方法 Unsupervised Data Generation，能够生成高质量的训练数据，无需人工标注。该方法使得用不带标签的数据训练具体任务的模型成为可能，并且当与带标签的数据混合使用时，能实现高效的数据增强，并达到了 SuperGLUE 基准测试的新最优结果。

零标签语言学习的探索

Towards Zero-Label Language Learning

In unsupervised data generation tasks, besides the generation of a sample
based on previous observations, one would often like to give hints to the model
in order to bias the generation towards desirable metrics. We propose a method
that combines Generative Adversarial Networks (GANs) and reinforcement learning
(RL) in order to accomplish exactly that. While RL biases the data generation
process towards arbitrary metrics, the GAN component of the reward function
ensures that the model still remembers information learned from data. We build
upon previous results that incorporated GANs and RL in order to generate
sequence data and test this model in several settings for the generation of
molecules encoded as text sequences (SMILES) and in the context of music
generation, showing for each case that we can effectively bias the generation
process towards desired metrics.

本研究提出了一种利用生成对抗网络和强化学习相结合的方法，实现对数据生成过程进行度量偏置，提高目标数据质量。通过在分子和音乐生成数据的生成过程中的应用，验证了该方法的有效性。