Large Language Models (LLMs) have demonstrated impressive zero shot
performance on a wide range of NLP tasks, demonstrating the ability to reason
and apply commonsense. A relevant application is to use them for creating high
quality synthetic datasets for downstream tasks. In this work, we probe whether
GPT-4 can be used to augment existing extractive reading comprehension
datasets. Automating data annotation processes has the potential to save large
amounts of time, money and effort that goes into manually labelling datasets.
In this paper, we evaluate the performance of GPT-4 as a replacement for human
annotators for low resource reading comprehension tasks, by comparing
performance after fine tuning, and the cost associated with annotation. This
work serves to be the first analysis of LLMs as synthetic data augmenters for
QA systems, highlighting the unique opportunities and challenges. Additionally,
we release augmented versions of low resource datasets, that will allow the
research community to create further benchmarks for evaluation of generated
datasets.

本文探讨了使用 GPT-4 作为人工标注的替代品来提供低资源阅读理解任务的性能，通过精细调节后的性能和标注成本的对比，这是对 LLMs 作为合成数据增广器用于 QA 系统的第一次分析，强调了这一独特的机遇和挑战，并提供了低资源数据集的增广版本，为生成数据集的评估提供了进一步的基准。