Well-annotated data is a prerequisite for good Natural Language Processing
models. Too often, though, annotation decisions are governed by optimizing time
or annotator agreement. We make a case for nuanced efforts in an
interdisciplinary setting for annotating offensive online speech. Detecting
offensive content is rapidly becoming one of the most important real-world NLP
tasks. However, most datasets use a single binary label, e.g., for hate or
incivility, even though each concept is multi-faceted. This modeling choice
severely limits nuanced insights, but also performance. We show that a more
fine-grained multi-label approach to predicting incivility and hateful or
intolerant content addresses both conceptual and performance issues. We release
a novel dataset of over 40,000 tweets about immigration from the US and UK,
annotated with six labels for different aspects of incivility and intolerance.
Our dataset not only allows for a more nuanced understanding of harmful speech
online, models trained on it also outperform or match performance on benchmark
datasets.

本文介绍了一种使用多个标签注释恶意在线言论的方法，强调了注释应该是细致、准确的。作者发布了一个高质量的数据集，使用六个标签注释了超过 40,000 条有关移民的推文，在此数据集上训练模型的表现优于基准数据集。