Large Language Models (LLMs), acting as a powerful reasoner and generator,
exhibit extraordinary performance across various natural language tasks, such
as question answering (QA). Among these tasks, Multi-Hop Question Answering
(MHQA) stands as a widely discussed category, necessitating seamless
integration between LLMs and the retrieval of external knowledge. Existing
methods employ LLM to generate reasoning paths and plans, and utilize IR to
iteratively retrieve related knowledge, but these approaches have inherent
flaws. On one hand, Information Retriever (IR) is hindered by the low quality
of generated queries by LLM. On the other hand, LLM is easily misguided by the
irrelevant knowledge by IR. These inaccuracies, accumulated by the iterative
interaction between IR and LLM, lead to a disaster in effectiveness at the end.
To overcome above barriers, in this paper, we propose a novel pipeline for MHQA
called Furthest-Reasoning-with-Plan-Assessment (FuRePA), including an improved
framework (Furthest Reasoning) and an attached module (Plan Assessor). 1)
Furthest reasoning operates by masking previous reasoning path and generated
queries for LLM, encouraging LLM generating chain of thought from scratch in
each iteration. This approach enables LLM to break the shackle built by
previous misleading thoughts and queries (if any). 2) The Plan Assessor is a
trained evaluator that selects an appropriate plan from a group of candidate
plans proposed by LLM. Our methods are evaluated on three highly recognized
public multi-hop question answering datasets and outperform state-of-the-art on
most metrics (achieving a 10%-12% in answer accuracy).

本文提出了一种针对多跳问题回答的新方法，称为 FuRePA，其中包括改进框架（Furthest Reasoning）和附加模块（Plan Assessor）。通过屏蔽之前的推理路径和生成的查询，鼓励 LLM 在每次迭代中从头开始生成思路，从而打破先前误导思想和查询的束缚。通过对三个公认的多跳问题回答数据集的评估，我们的方法在大多数指标上优于现有技术（回答准确率提高了 10%-12%）。

利用计划评估进行最远推理：检索增强的大型语言模型稳定推理路径

Furthest Reasoning with Plan Assessment: Stable Reasoning Path with  Retrieval-Augmented Large Language Models

Information Retriever (IR) aims to find the relevant documents (e.g.
snippets, passages, and articles) to a given query at large scale. IR plays an
important role in many tasks such as open domain question answering and
dialogue systems, where external knowledge is needed. In the past, searching
algorithms based on term matching have been widely used. Recently, neural-based
algorithms (termed as neural retrievers) have gained more attention which can
mitigate the limitations of traditional methods. Regardless of the success
achieved by neural retrievers, they still face many challenges, e.g. suffering
from a small amount of training data and failing to answer simple
entity-centric questions. Furthermore, most of the existing neural retrievers
are developed for pure-text query. This prevents them from handling
multi-modality queries (i.e. the query is composed of textual description and
images). This proposal has two goals. First, we introduce methods to address
the abovementioned issues of neural retrievers from three angles, new model
architectures, IR-oriented pretraining tasks, and generating large scale
training data. Second, we identify the future research direction and propose
potential corresponding solution.

本篇论文提出三个方面的方法，包括新的模型架构、IR 导向的预训练任务、生成大规模训练数据，以解决神经检索器面临的问题，并探讨了多模式查询的未来研究方向和相应的解决方法。