Social media platforms are deploying machine learning based offensive language classification systems to combat hateful, racist, and other forms of offensive speech at scale. However, despite their real-world deployment, we do not yet comprehensively understand the extent to which offensive language classifiers are robust against adversarial attacks. Prior work in this space is limited to studying robustness of offensive language classifiers against primitive attacks such as misspellings and extraneous spaces. To address this gap, we systematically analyze the robustness of state-of-the-art offensive language classifiers against more crafty adversarial attacks that leverage greedy- and attention-based word selection and context-aware embeddings for word replacement. Our results on multiple datasets show that these crafty adversarial attacks can degrade the accuracy of offensive language classifiers by more than 50% while also being able to preserve the readability and meaning of the modified text.

该研究对社交媒体平台上的机器学习型进攻性语言分类器的鲁棒性做出了系统的分析，并证明了具有贪婪和注意力机制的词汇选择和上下文感知嵌入的攻击可将这些分类器的准确性降低50%以上，同时还能保持修改后文本的可读性和含义。

论攻击性语言分类器的鲁棒性