While diverse question answering (QA) datasets have been proposed and
contributed significantly to the development of deep learning models for QA
tasks, the existing datasets fall short in two aspects. First, we lack QA
datasets covering complex questions that involve answers as well as the
reasoning processes to get the answers. As a result, the state-of-the-art QA
research on numerical reasoning still focuses on simple calculations and does
not provide the mathematical expressions or evidences justifying the answers.
Second, the QA community has contributed much effort to improving the
interpretability of QA models. However, these models fail to explicitly show
the reasoning process, such as the evidence order for reasoning and the
interactions between different pieces of evidence. To address the above
shortcomings, we introduce NOAHQA, a conversational and bilingual QA dataset
with questions requiring numerical reasoning with compound mathematical
expressions. With NOAHQA, we develop an interpretable reasoning graph as well
as the appropriate evaluation metric to measure the answer quality. We evaluate
the state-of-the-art QA models trained using existing QA datasets on NOAHQA and
show that the best among them can only achieve 55.5 exact match scores, while
the human performance is 89.7. We also present a new QA model for generating a
reasoning graph where the reasoning graph metric still has a large gap compared
with that of humans, e.g., 28 scores.

介绍一个双语问答数据集 NOAHQA，旨在解决目前数值推理问题缺乏复杂问题以及提供推理过程的可解释性问题，通过使用 NOAHQA 数据集，开发了一个可解释的推理图，并提出了适当的评估指标来衡量答案质量。在 NOAHQA 上对现有 QA 模型进行评估，表明人类表现为 89.7，最好的 QA 模型只能达到 55.5 的准确匹配分数。