Warning: This paper contains content that may be offensive or upsetting. Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which statements are made. For example, the utterance "your English is very good" may implicitly signal an insult when uttered by a white man to a non-white colleague, but uttered by an ESL teacher to their student would be interpreted as a genuine compliment. Such contextual factors have been largely ignored by previous approaches to toxic language detection. We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements grounded in their social and situational context. We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts and free-text explanations of offensiveness, implied biases, speaker intents, and listener reactions. To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context. We find that explanations by context-agnostic models are significantly worse than by context-aware ones, especially in situations where the context inverts the statement's offensiveness (29% accuracy drop). Our work highlights the importance and feasibility of contextualized NLP by modeling social factors.

本文提出了 COBRA 框架，这是第一种上下文感知的形式主义，用于解释有害或有偏见言论的意图、反应和危害，着重于其社会和情境背景。我们创建了 COBRACORPUS 数据集，并发现上下文不敏感模型的解释显着劣于上下文感知模型，特别是在上下文反转言论的冒犯性时。本研究强调了上下文化 NLP 建模社会因素的重要性和可行性。

COBRA框架：有关攻击性言论的影响和伤害的情境推理