Computational social science (CSS) practitioners often rely on human-labeled
data to fine-tune supervised text classifiers. We assess the potential for
researchers to augment or replace human-generated training data with surrogate
training labels from generative large language models (LLMs). We introduce a
recommended workflow and test this LLM application by replicating 14
classification tasks and measuring performance. We employ a novel corpus of
English-language text classification data sets from recent CSS articles in
high-impact journals. Because these data sets are stored in password-protected
archives, our analyses are less prone to issues of contamination. For each
task, we compare supervised classifiers fine-tuned using GPT-4 labels against
classifiers fine-tuned with human annotations and against labels from GPT-4 and
Mistral-7B with few-shot in-context learning. Our findings indicate that
supervised classification models fine-tuned on LLM-generated labels perform
comparably to models fine-tuned with labels from human annotators. Fine-tuning
models using LLM-generated labels can be a fast, efficient and cost-effective
method of building supervised text classifiers.

使用生成的大型语言模型生成的标签对监督文本分类模型进行微调，与使用人工标注的标签相比表现相当，是一种快速、高效和经济有效的构建监督文本分类器的方法。