In this paper, we investigate which questions are challenging for retrieval-based Question Answering (QA). We (i) propose retrieval complexity (RC), a novel metric conditioned on the completeness of retrieved documents, which measures the difficulty of answering questions, and (ii) propose an unsupervised pipeline to measure RC given an arbitrary retrieval system. Our proposed pipeline measures RC more accurately than alternative estimators, including LLMs, on six challenging QA benchmarks. Further investigation reveals that RC scores strongly correlate with both QA performance and expert judgment across five of the six studied benchmarks, indicating that RC is an effective measure of question difficulty. Subsequent categorization of high-RC questions shows that they span a broad set of question shapes, including multi-hop, compositional, and temporal QA, indicating that RC scores can categorize a new subset of complex questions. Our system can also have a major impact on retrieval-based systems by helping to identify more challenging questions on existing datasets.

该研究通过提出检索复杂度（RC）指标以及一个无监督的测量RC的流程，较准确地衡量了检索式问答任务中问题的难度，并发现RC与问答系统性能和专家判断之间有很强的相关性，可以对复杂问题进行分类。这一系统还可以通过确定现有数据集中的更具挑战性的问题，对检索式系统产生重大影响。

问题回答系统中检测检索复杂度