Complex question answering over knowledge base (Complex KBQA) is challenging
because it requires various compositional reasoning capabilities, such as
multi-hop inference, attribute comparison, set operation. Existing benchmarks
have some shortcomings that limit the development of Complex KBQA: 1) they only
provide QA pairs without explicit reasoning processes; 2) questions are poor in
diversity or scale. To this end, we introduce KQA Pro, a dataset for Complex
KBQA including ~120K diverse natural language questions. We introduce a
compositional and interpretable programming language KoPL to represent the
reasoning process of complex questions. For each question, we provide the
corresponding KoPL program and SPARQL query, so that KQA Pro serves for both
KBQA and semantic parsing tasks. Experimental results show that SOTA KBQA
methods cannot achieve promising results on KQA Pro as on current datasets,
which suggests that KQA Pro is challenging and Complex KBQA requires further
research efforts. We also treat KQA Pro as a diagnostic dataset for testing
multiple reasoning skills, conduct a thorough evaluation of existing models and
discuss further directions for Complex KBQA. Our codes and datasets can be
obtained from this https URL

本文介绍了一个题库 KQA Pro，旨在解决现有题库对多复杂推理能力的不足，采用了一种可组合且可解释的编程语言 KoPL 来表示复杂问题的推理过程，提供了与每个问题对应的 KoPL 程序和 SPARQL 查询，可以用于知识库问答和语义解析任务，并将其视为测试多种推理能力的诊断性数据集。实验结果表明，SOTA KBQA 方法在 KQA Pro 上无法取得与当前数据集同样的优异结果，这表明 KQA Pro 很具有挑战性，需要进一步的研究努力。