AI has achieved remarkable mastery over games such as Chess, Go, and Poker,
and even Jeopardy, but the rich variety of standardized exams has remained a
landmark challenge. Even in 2016, the best AI system achieved merely 59.3% on
an 8th Grade science exam challenge. This paper reports unprecedented success
on the Grade 8 New York Regents Science Exam, where for the first time a system
scores more than 90% on the exam's non-diagram, multiple choice (NDMC)
questions. In addition, our Aristo system, building upon the success of recent
language models, exceeded 83% on the corresponding Grade 12 Science Exam NDMC
questions. The results, on unseen test questions, are robust across different
test years and different variations of this kind of test. They demonstrate that
modern NLP methods can result in mastery on this task. While not a full
solution to general question-answering (the questions are multiple choice, and
the domain is restricted to 8th Grade science), it represents a significant
milestone for the field.

本研究报告了 Aristo 系统在 8 年级纽约 Regents 科学考试和 12 年级的科学考试中，使用 NLP 方法在非图表多项选择题 (NDMC) 上均实现了历史性突破，证明了现代自然语言处理方法在一定程度上能够掌握此类任务。