Open-domain complex Question Answering (QA) is a difficult task with
challenges in evidence retrieval and reasoning. The complexity of such
questions could stem from questions being compositional, hybrid evidence, or
ambiguity in questions. While retrieval performance for classical QA tasks is
well explored, their capabilities for heterogeneous complex retrieval tasks,
especially in an open-domain setting, and the impact on downstream QA
performance, are relatively unexplored. To address this, in this work, we
propose a benchmark composing diverse complex QA tasks and provide a toolkit to
evaluate state-of-the-art pre-trained dense and sparse retrieval models in an
open-domain setting. We observe that late interaction models and surprisingly
lexical models like BM25 perform well compared to other pre-trained dense
retrieval models. In addition, since context-based reasoning is critical for
solving complex QA tasks, we also evaluate the reasoning capabilities of LLMs
and the impact of retrieval performance on their reasoning capabilities.
Through experiments, we observe that much progress is to be made in retrieval
for complex QA to improve downstream QA performance. Our software and related
data can be accessed at this https URL

通过建立一个包含多样化复杂问答任务的基准以及提供工具包，在开放领域环境中评估先进的预训练密集和稀疏检索模型，本研究旨在解决开放领域复杂问答中证据检索和推理的挑战。我们观察到晚期交互模型和词汇模型（如 BM25）表现较好，此外，我们还评估了 LLMs 的推理能力以及检索性能对其推理能力的影响，通过实验证实，要改进复杂问答的下游性能，还有很多待办之事。