This work investigates the use of interactively updated label suggestions to
improve upon the efficiency of gathering annotations on the task of opinion
mining in German Covid-19 social media data. We develop guidelines to conduct a
controlled annotation study with social science students and find that
suggestions from a model trained on a small, expert-annotated dataset already
lead to a substantial improvement - in terms of inter-annotator agreement(+.14
Fleiss' $\kappa$) and annotation quality - compared to students that do not
receive any label suggestions. We further find that label suggestions from
interactively trained models do not lead to an improvement over suggestions
from a static model. Nonetheless, our analysis of suggestion bias shows that
annotators remain capable of reflecting upon the suggested label in general.
Finally, we confirm the quality of the annotated data in transfer learning
experiments between different annotator groups. To facilitate further research
in opinion mining on social media data, we release our collected data
consisting of 200 expert and 2,785 student annotations.

本文研究了使用交互式更新标签建议来改善在德国新冠社交媒体数据的意见挖掘任务中获取注释的效率。作者开发了一些指导方针并进行了一项受控注释研究，发现模型从一个小型专家注释数据集中训练的建议已经可以显著提高注释者的质量，而来自交互式训练模型的标签建议并没有比静态模型带来改进。