Large language models (LLMs) effectively generate fluent text when the target
output follows natural language patterns. However, structured prediction tasks
confine the output format to a limited ontology, causing even very large models
to struggle since they were never trained with such restrictions in mind. The
difficulty of using LLMs for direct prediction is exacerbated in few-shot
learning scenarios, which commonly arise due to domain shift and resource
limitations. We flip the problem on its head by leveraging the LLM as a tool
for data augmentation rather than direct prediction. Our proposed Mixture of
Soft Prompts (MSP) serves as a parameter-efficient procedure for generating
data in a controlled manner. Denoising mechanisms are further applied to
improve the quality of synthesized data. Automatic metrics show our method is
capable of producing diverse and natural text, while preserving label
semantics. Moreover, MSP achieves state-of-the-art results on three benchmarks
when compared against strong baselines. Our method offers an alternate
data-centric approach for applying LLMs to complex prediction tasks.

本文介绍了一种利用大型语言模型作为数据增强工具的技术，即利用软提示的混合模型进行参数有效的数据生成，并通过去噪机制提高生成数据的质量，此方法能够在复杂预测任务中实现对标签语义的保留，达到优于强基准的最新结果。