Complex Query Answering (CQA) is an important reasoning task on knowledge
graphs. Current CQA learning models have been shown to be able to generalize
from atomic operators to more complex formulas, which can be regarded as the
combinatorial generalizability. In this paper, we present EFO-1-QA, a new
dataset to benchmark the combinatorial generalizability of CQA models by
including 301 different queries types, which is 20 times larger than existing
datasets. Besides, our work, for the first time, provides a benchmark to
evaluate and analyze the impact of different operators and normal forms by
using (a) 7 choices of the operator systems and (b) 9 forms of complex queries.
Specifically, we provide the detailed study of the combinatorial
generalizability of two commonly used operators, i.e., projection and
intersection, and justify the impact of the forms of queries given the
canonical choice of operators. Our code and data can provide an effective
pipeline to benchmark CQA models.

本文提出 EFO-1-QA 数据集，包含 301 种不同的查询类型用于评估目前复杂查询回答 (CQA) 学习模型的组合泛化能力，并提供可能影响查询泛化能力的不同操作系统和形式的分析和评估。