Stance detection is an important task for many applications that analyse or
support online political discussions. Common approaches include fine-tuning
transformer based models. However, these models require a large amount of
labelled data, which might not be available. In this work, we present two
different ways to leverage LLM-generated synthetic data to train and improve
stance detection agents for online political discussions: first, we show that
augmenting a small fine-tuning dataset with synthetic data can improve the
performance of the stance detection model. Second, we propose a new active
learning method called SQBC based on the "Query-by-Comittee" approach. The key
idea is to use LLM-generated synthetic data as an oracle to identify the most
informative unlabelled samples, that are selected for manual labelling.
Comprehensive experiments show that both ideas can improve the stance detection
performance. Curiously, we observed that fine-tuning on actively selected
samples can exceed the performance of using the full dataset.

通过使用 LLM 生成的合成数据，我们提供了两种不同的方法来训练和改进在线政治讨论的立场检测代理：一是通过合成数据增加微调数据集以改善立场检测模型的性能；二是基于 “Query-by-Committee” 方法提出了一种新的主动学习方法 SQBC，利用 LLM 生成的合成数据作为预测标准，选择最具信息量的未标记样本进行手动标记。综合实验证明，这两种方法都能改善立场检测性能。有趣的是，我们观察到在主动选择样本上微调可以超过使用完整数据集的性能。

SQBC: 在线政治讨论中基于 LLM 生成的合成数据的立场检测的主动学习

SQBC: Active Learning using LLM-Generated Synthetic Data for Stance  Detection in Online Political Discussions

Deep neural networks usually require large labeled datasets for training to
achieve state-of-the-art performance in many tasks, such as image
classification and natural language processing. Although a lot of data is
created each day by active Internet users, most of these data are unlabeled and
are vulnerable to data poisoning attacks. In this paper, we develop an
efficient active learning method that requires fewer labeled instances and
incorporates the technique of adversarial retraining in which additional
labeled artificial data are generated without increasing the budget of the
labeling. The generated adversarial examples also provide a way to measure the
vulnerability of the model. To check the performance of the proposed method
under an adversarial setting, i.e., malicious mislabeling and data poisoning
attacks, we perform an extensive evaluation on the reduced CIFAR-10 dataset,
which contains only two classes: airplane and frog. Our experimental results
demonstrate that the proposed active learning method is efficient for defending
against malicious mislabeling and data poisoning attacks. Specifically, whereas
the baseline active learning method based on the random sampling strategy
performs poorly (about 50%) under a malicious mislabeling attack, the proposed
active learning method can achieve the desired accuracy of 89% using only
one-third of the dataset on average.

本文介绍了一种高效的主动学习方法，它结合了敌对再训练技术，可以生成更多的人工标注数据集而不增加标注预算，产生的敌对样本也提供了一种测量模型易受攻击的方式。作者在减小的 CIFAR-10 数据集上对其性能进行了充分评估，得出该方法有效对抗恶意袭击的结论。