This paper proposes a question-answering system that can answer questions
whose supporting evidence is spread over multiple (potentially long) documents.
The system, called Visconde, uses a three-step pipeline to perform the task:
decompose, retrieve, and aggregate. The first step decomposes the question into
simpler questions using a few-shot large language model (LLM). Then, a
state-of-the-art search engine is used to retrieve candidate passages from a
large collection for each decomposed question. In the final step, we use the
LLM in a few-shot setting to aggregate the contents of the passages into the
final answer. The system is evaluated on three datasets: IIRC, Qasper, and
StrategyQA. Results suggest that current retrievers are the main bottleneck and
that readers are already performing at the human level as long as relevant
passages are provided. The system is also shown to be more effective when the
model is induced to give explanations before answering a question. Code is
available at https://github.com/neuralmind-ai/visconde.

本文提出了一个名为 Visconde 的问答系统，利用 few-shot 大型语言模型和先进的搜索引擎检索多个文档中的支持证据用于回答问题，评估结果表明检索机制是主要的瓶颈，为了得到更好的回答表现，模型在回答问题前给出解释是有效的。