A multi-hop question answering (QA) dataset aims to test reasoning and
inference skills by requiring a model to read multiple paragraphs to answer a
given question. However, current datasets do not provide a complete explanation
for the reasoning process from the question to the answer. Further, previous
studies revealed that many examples in existing multi-hop datasets do not
require multi-hop reasoning to answer a question. In this study, we present a
new multi-hop QA dataset, called 2WikiMultiHopQA, which uses structured and
unstructured data. In our dataset, we introduce the evidence information
containing a reasoning path for multi-hop questions. The evidence information
has two benefits: (i) providing a comprehensive explanation for predictions and
(ii) evaluating the reasoning skills of a model. We carefully design a pipeline
and a set of templates when generating a question-answer pair that guarantees
the multi-hop steps and the quality of the questions. We also exploit the
structured format in Wikidata and use logical rules to create questions that
are natural but still require multi-hop reasoning. Through experiments, we
demonstrate that our dataset is challenging for multi-hop models and it ensures
that multi-hop reasoning is required.

本研究创建了一个名为 2WikiMultiHopQA 的新的多跳问题回答数据集，使用结构化和非结构化数据生成问题 - 答案对，并提供包含推理路径的证据信息以提供预测的全面解释并评估模型推理技能。通过实验，我们证明了我们的数据集对于多跳模型来说具有挑战性，并确保需要使用多跳推理。