Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. Ablation studies and a qualitative analysis provide more insights into our approach.

本文提出一种新的数据增广技术，利用大规模的语言模型从混合的样本中生成逼真的文本样本，并利用语言模型预测的软标签，从大规模语言模型中蒸馏知识并同时创建文本扰动，我们在多元化的分类任务上进行数据增广实验，并展示了该方法远远优于现有的文本增广方法，消融研究和定性分析提供了更多的见解。

GPT3Mix：利用大规模语言模型进行文本增强