To answer complex queries on knowledge graphs, logical reasoning over
incomplete knowledge is required due to the open-world assumption.
Learning-based methods are essential because they are capable of generalizing
over unobserved knowledge. Therefore, an appropriate dataset is fundamental to
both obtaining and evaluating such methods under this paradigm. In this paper,
we propose a comprehensive framework for data generation, model training, and
method evaluation that covers the combinatorial space of Existential
First-order Queries with multiple variables ($\text{EFO}_{k}$). The
combinatorial query space in our framework significantly extends those defined
by set operations in the existing literature. Additionally, we construct a
dataset, $\text{EFO}_{k}$-CQA, with 741 types of query for empirical
evaluation, and our benchmark results provide new insights into how query
hardness affects the results. Furthermore, we demonstrate that the existing
dataset construction process is systematically biased that hinders the
appropriate development of query-answering methods, highlighting the importance
of our work. Our code and data are provided
in~https://github.com/HKUST-KnowComp/EFOK-CQA.

为了回答知识图谱上的复杂查询，我们提出了一个综合框架，涵盖了存在性一阶查询的组合空间，并构建了一个 dataset 进行评估。我们的结果揭示了查询难度对结果的影响，并强调了现有数据集构建过程中的系统偏见对查询 - 回答方法的适当发展的重要性。