Though majority vote among annotators is typically used for ground truth
labels in natural language processing, annotator disagreement in tasks such as
hate speech detection may reflect differences among group opinions, not noise.
Thus, a crucial problem in hate speech detection is whether a statement is
offensive to the demographic group that it targets, which may constitute a
small fraction of the annotator pool. We construct a model that predicts
individual annotator ratings on potentially offensive text and combines this
information with the predicted target group of the text to model the opinions
of target group members. We show gains across a range of metrics, including
raising performance over the baseline by 22% at predicting individual
annotators' ratings and 33% at predicting variance among annotators, which
provides a method of measuring model uncertainty downstream. We find that
annotators' ratings can be predicted using their demographic information and
opinions on online content, without the need to track identifying annotator IDs
that link each annotator to their ratings. We also find that use of
non-invasive survey questions on annotators' online experiences helps to
maximize privacy and minimize unnecessary collection of demographic information
when predicting annotators' opinions.

针对网络暴力检测中的数据标注员意见分歧问题，提出了一种预测个体标注员评估与目标群体意见的模型，并通过结合潜在目标群体与评估进行评估，在预测任务中表现出了 22% 的性能提升和 33% 的模型不确定性预测能力。我们发现，标注员的评估可以通过其在线内容的人口统计信息和意见进行预测，而无需跟踪标注员 ID。同时，我们还发现在预测标注员意见时，使用非侵入性调查问题有助于最大程度保护隐私和最小化不必要的个人信息收集。