In this paper, we investigate the use of data obtained from prompting a large
generative language model, ChatGPT, to generate synthetic training data with
the aim of augmenting data in low resource scenarios. We show that with
appropriate task-specific ChatGPT prompts, we outperform the most popular
existing approaches for such data augmentation. Furthermore, we investigate
methodologies for evaluating the similarity of the augmented data generated
from ChatGPT with the aim of validating and assessing the quality of the data
generated.

利用生成式语言模型 ChatGPT 生成合成训练数据，以增强低资源场景下的数据，我们使用特定任务的 ChatGPT 提示，表现优于现有方法，并研究了评估合成数据相似性的方法，以验证和评估所生成数据的质量。

使用 ChatGPT 生成和增强训练数据的 ZeroShotDataAug

ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT

Neural sequence labeling (NSL) aims at assigning labels for input language
tokens, which covers a broad range of applications, such as named entity
recognition (NER) and slot filling, etc. However, the satisfying results
achieved by traditional supervised-based approaches heavily depend on the large
amounts of human annotation data, which may not be feasible in real-world
scenarios due to data privacy and computation efficiency issues. This paper
presents SeqUST, a novel uncertain-aware self-training framework for NSL to
address the labeled data scarcity issue and to effectively utilize unlabeled
data. Specifically, we incorporate Monte Carlo (MC) dropout in Bayesian neural
network (BNN) to perform uncertainty estimation at the token level and then
select reliable language tokens from unlabeled data based on the model
confidence and certainty. A well-designed masked sequence labeling task with a
noise-robust loss supports robust training, which aims to suppress the problem
of noisy pseudo labels. In addition, we develop a Gaussian-based consistency
regularization technique to further improve the model robustness on
Gaussian-distributed perturbed representations. This effectively alleviates the
over-fitting dilemma originating from pseudo-labeled augmented data. Extensive
experiments over six benchmarks demonstrate that our SeqUST framework
effectively improves the performance of self-training, and consistently
outperforms strong baselines by a large margin in low-resource scenarios

本文提出了 SeqUST， 一种新颖的不确定性感知自我训练框架，以解决标记数据稀缺性问题并有效利用未标记数据来改善神经序列标注的性能，并在六个基准测试中展现出在低资源场景中超越强基线模型的能力。