Collecting annotations from human raters often results in a trade-off between
the quantity of labels one wishes to gather and the quality of these labels. As
such, it is often only possible to gather a small amount of high-quality
labels. In this paper, we study how different training strategies can leverage
a small dataset of human-annotated labels and a large but noisy dataset of
synthetically generated labels (which exhibit bias against identity groups) for
predicting toxicity of online comments. We evaluate the accuracy and fairness
properties of these approaches, and trade-offs between the two. While we find
that initial training on all of the data and fine-tuning on clean data produces
models with the highest AUC, we find that no single strategy performs best
across all fairness metrics.

本研究探讨不同训练策略如何利用少量人工注释标签和大量但带有偏见的合成标签（针对身份群体）来预测在线评论的毒性，并评估了这些方法的准确性和公正性。虽然最初使用所有数据进行训练并在干净数据上微调能够产生具有最高 AUC 的模型，但同时我们发现没有一种策略在所有公正度量标准上表现最佳。