The rapid advancement of large language models (LLMs) has sparked interest in
data synthesis techniques, aiming to generate diverse and high-quality
synthetic datasets. However, these synthetic datasets often suffer from a lack
of diversity and added noise. In this paper, we present TarGEN, a multi-step
prompting strategy for generating high-quality synthetic datasets utilizing a
LLM. An advantage of TarGEN is its seedless nature; it does not require
specific task instances, broadening its applicability beyond task replication.
We augment TarGEN with a method known as self-correction empowering LLMs to
rectify inaccurately labeled instances during dataset creation, ensuring
reliable labels. To assess our technique's effectiveness, we emulate 8 tasks
from the SuperGLUE benchmark and finetune various language models, including
encoder-only, encoder-decoder, and decoder-only models on both synthetic and
original training sets. Evaluation on the original test set reveals that models
trained on datasets generated by TarGEN perform approximately 1-2% points
better than those trained on original datasets (82.84% via syn. vs. 81.12% on
og. using Flan-T5). When incorporating instruction tuning, the performance
increases to 84.54% on synthetic data vs. 81.49% on original data by Flan-T5. A
comprehensive analysis of the synthetic dataset compared to the original
dataset reveals that the synthetic dataset demonstrates similar or higher
levels of dataset complexity and diversity. Furthermore, the synthetic dataset
displays a bias level that aligns closely with the original dataset. Finally,
when pre-finetuned on our synthetic SuperGLUE dataset, T5-3B yields impressive
results on the OpenLLM leaderboard, surpassing the model trained on the
Self-Instruct dataset by 4.14% points. We hope that TarGEN can be helpful for
quality data generation and reducing the human efforts to create complex
benchmarks.

通过使用 TarGEN 多步提示策略来生成高质量的合成数据集，结合自校正方法增强语言模型生成过程中的准确性，从而创建复杂和多样的基准测试数据集。

TarGEN：大型语言模型的有针对性数据生成

TarGEN: Targeted Data Generation with Large Language Models

We present a state-of-the-art speech recognition system developed using
end-to-end deep learning. Our architecture is significantly simpler than
traditional speech systems, which rely on laboriously engineered processing
pipelines; these traditional systems also tend to perform poorly when used in
noisy environments. In contrast, our system does not need hand-designed
components to model background noise, reverberation, or speaker variation, but
instead directly learns a function that is robust to such effects. We do not
need a phoneme dictionary, nor even the concept of a "phoneme." Key to our
approach is a well-optimized RNN training system that uses multiple GPUs, as
well as a set of novel data synthesis techniques that allow us to efficiently
obtain a large amount of varied data for training. Our system, called Deep
Speech, outperforms previously published results on the widely studied
Switchboard Hub5'00, achieving 16.0% error on the full test set. Deep Speech
also handles challenging noisy environments better than widely used,
state-of-the-art commercial speech systems.

使用端到端深度学习开发的演讲识别系统：不需要手工设计组件来模拟背景噪声、混响或者发言人差异，取而代之的是直接学习一个对这些影响具有鲁棒性的函数的方法，其关键是优化的 RNN 训练系统以及一组新的数据综合技术，该系统在 Switchboard Hub5'00 测试集上取得了 16.0% 的错误率，优于以前的成果，并且与广泛使用的最先进的商业演讲系统相比，Deep Speech 处理具有挑战性的嘈杂环境的能力更加出色。