We test whether a leading AI system GPT-4 understands discourse as well as
humans do, using a standardized test of discourse comprehension. Participants
are presented with brief stories and then answer eight yes/no questions probing
their comprehension of the story. The questions are formatted to assess the
separate impacts of directness (stated vs. implied) and salience (main idea vs.
details). GPT-4 performs slightly, but not statistically significantly, better
than humans given the very high level of human performance. Both GPT-4 and
humans exhibit a strong ability to make inferences about information that is
not explicitly stated in a story, a critical test of understanding.

GPT-4 与人类在标准化的议论理解测试中表现相近，都对故事中暗含的信息具有强大的推理能力。

GPT-4 在至少跟人类一样理解语篇

GPT-4 Understands Discourse at Least as Well as Humans Do

While there has been substantial progress in text comprehension through
simple factoid question answering, more holistic comprehension of a discourse
still presents a major challenge (Dunietz et al., 2020). Someone critically
reflecting on a text as they read it will pose curiosity-driven, often
open-ended questions, which reflect deep understanding of the content and
require complex reasoning to answer (Ko et al., 2020; Westera et al., 2020). A
key challenge in building and evaluating models for this type of discourse
comprehension is the lack of annotated data, especially since collecting
answers to such questions requires high cognitive load for annotators. This
paper presents a novel paradigm that enables scalable data collection targeting
the comprehension of news documents, viewing these questions through the lens
of discourse. The resulting corpus, DCQA (Discourse Comprehension by Question
Answering), captures both discourse and semantic links between sentences in the
form of free-form, open-ended questions. On an evaluation set that we annotated
on questions from Ko et al. (2020), we show that DCQA provides valuable
supervision for answering open-ended questions. We additionally design
pre-training methods utilizing existing question-answering resources, and use
synthetic data to accommodate unanswerable questions.

本文提出了一种新的方法来收集关于新闻文件的提问，带有上下文关联且没有标注答案的开放性问题数据集 (DCQA)，并结合现有的问答资源进行了预训练，以有效帮助回答这类开放性问题。