We introduce STREET, a unified multi-task and multi-domain natural language
reasoning and explanation benchmark. Unlike most existing question-answering
(QA) datasets, we expect models to not only answer questions, but also produce
step-by-step structured explanations describing how premises in the question
are used to produce intermediate conclusions that can prove the correctness of
a certain answer. We perform extensive evaluation with popular language models
such as few-shot prompting GPT-3 and fine-tuned T5. We find that these models
still lag behind human performance when producing such structured reasoning
steps. We believe this work will provide a way for the community to better
train and test systems on multi-step reasoning and explanations in natural
language.

本文介绍 STREET，一个统一的多任务、多领域自然语言推理和解释基准。该基准要求模型不仅能回答问题，还能产生步骤化的结构化解释，并描述问题中的前提是如何用于产生能证明某个答案正确性的中间结论。作者使用 GPT-3 和 T5 等常见语言模型进行了广泛的评估，发现这些模型在产生这些结构化推理步骤方面仍远落后于人类表现。作者相信，这项工作将为社区提供一种更好的方法，以在自然语言的多步推理和解释方面进行更好的系统培训和测试。