The full power of human language-based communication cannot be realized
without negation. All human languages have some form of negation. Despite this,
negation remains a challenging phenomenon for current natural language
understanding systems. To facilitate the future development of models that can
process negation effectively, we present CONDAQA, the first English reading
comprehension dataset which requires reasoning about the implications of
negated statements in paragraphs. We collect paragraphs with diverse negation
cues, then have crowdworkers ask questions about the implications of the
negated statement in the passage. We also have workers make three kinds of
edits to the passage -- paraphrasing the negated statement, changing the scope
of the negation, and reversing the negation -- resulting in clusters of
question-answer pairs that are difficult for models to answer with spurious
shortcuts. CONDAQA features 14,182 question-answer pairs with over 200 unique
negation cues and is challenging for current state-of-the-art models. The best
performing model on CONDAQA (UnifiedQA-v2-3b) achieves only 42% on our
consistency metric, well below human performance which is 81%. We release our
dataset, along with fully-finetuned, few-shot, and zero-shot evaluations, to
facilitate the development of future NLP methods that work on negated language.

该研究介绍了 CONDAQA，这是第一个要求关于段落中否定语句涵义的英语阅读理解数据集，它挑战现有的自然语言理解系统，具有多样化的否定提示符，重点精心设计了具有谐音、范围变化和否定转变的三种编辑方式，细致构建了一系列难以回答的问题 - 答案对来测试模型的性能。最终的 CONDAQA 包含 14182 个问题 - 答案对及 200 多个唯一的否定提示符，最好的模型 (UnifiedQA-v2-3b) 仅在一致性测度上达到 42% 的表现，远低于人类表现 (81%)，因此研究团队希望将他们的数据集、每个问题答案的完整编辑和全面微调发布出来，以促进未来针对否定语言的 NLP 方法的发展。