Recently, dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). The final task-specific model often achieves compatible or even better performance than PLMs under the zero-shot setting, with orders of magnitude fewer parameters. However, synthetic datasets have their drawbacks. They have long been suffering from low-quality issues (e.g., low informativeness and redundancy). This explains why the massive synthetic data does not lead to better performance -- a scenario we would expect in the human-labeled data. To improve the quality of dataset synthesis, we propose a progressive zero-shot dataset generation framework, ProGen, which leverages the feedback from the task-specific model to guide the generation of new training data via in-context examples. Extensive experiments on five text classification datasets demonstrate the effectiveness of the proposed approach. We also show ProGen achieves on-par or superior performance with only 1\% synthetic dataset size compared to baseline methods without in-context feedback.

本文提出了一种渐进式的零样本数据集生成框架（ProGen），该框架利用任务特定模型的反馈通过上下文示例来指导生成新的训练数据，实验结果表明该方法在五个文本分类数据集上有效，并与没有上下文反馈的基线方法相比，只有1％的合成数据大小就可以达到与其相当或更好的性能。

ProGen: 利用上下文反馈逐步实现零样本数据集生成