Although the Retrieval-Augmented Generation (RAG) paradigms can use external
knowledge to enhance and ground the outputs of Large Language Models (LLMs) to
mitigate generative hallucinations and static knowledge base problems, they
still suffer from limited flexibility in adopting Information Retrieval (IR)
systems with varying capabilities, constrained interpretability during the
multi-round retrieval process, and a lack of end-to-end optimization. To
address these challenges, we propose a novel LLM-centric approach, IM-RAG, that
integrates IR systems with LLMs to support multi-round RAG through learning
Inner Monologues (IM, i.e., the human inner voice that narrates one's
thoughts). During the IM process, the LLM serves as the core reasoning model
(i.e., Reasoner) to either propose queries to collect more information via the
Retriever or to provide a final answer based on the conversational context. We
also introduce a Refiner that improves the outputs from the Retriever,
effectively bridging the gap between the Reasoner and IR modules with varying
capabilities and fostering multi-round communications. The entire IM process is
optimized via Reinforcement Learning (RL) where a Progress Tracker is
incorporated to provide mid-step rewards, and the answer prediction is further
separately optimized via Supervised Fine-Tuning (SFT). We conduct extensive
experiments with the HotPotQA dataset, a popular benchmark for retrieval-based,
multi-step question-answering. The results show that our approach achieves
state-of-the-art (SOTA) performance while providing high flexibility in
integrating IR modules as well as strong interpretability exhibited in the
learned inner monologues.

通过集成 IR 系统与 LLMs 来支持多轮 RAG，提出了一种新的 LLM 中心方法 IM-RAG，优化通过强化学习和监督微调，实现在 HotPotQA 数据集上达到最先进性能的同时提供高度可灵活性的 IR 模块集成和学习的内部独白解释性。