General Question Answering (QA) systems over texts require the multi-hop
reasoning capability, i.e. the ability to reason with information collected
from multiple passages to derive the answer. In this paper we conduct a
systematic analysis to assess such an ability of various existing models
proposed for multi-hop QA tasks. Specifically, our analysis investigates that
whether providing the full reasoning chain of multiple passages, instead of
just one final passage where the answer appears, could improve the performance
of the existing QA models. Surprisingly, when using the additional evidence
passages, the improvements of all the existing multi-hop reading approaches are
rather limited, with the highest error reduction of 5.8% on F1 (corresponding
to 1.3% absolute improvement) from the BERT model.
To better understand whether the reasoning chains could indeed help find
correct answers, we further develop a co-matching-based method that leads to
13.1% error reduction with passage chains when applied to two of our base
readers (including BERT). Our results demonstrate the existence of the
potential improvement using explicit multi-hop reasoning and the necessity to
develop models with better reasoning abilities.

本文系统分析了多篇文章 QA 任务的多跳推理能力，在提供额外证据段落的情况下，对现有面向多跳阅读的方法进行了改进，并提出了基于匹配的方法，表明进行显示多跳推理可以提高 QA 任务的执行效果，推荐开发更好的推理模型。