Retrieval-augmented generation integrates the capabilities of large language
models with relevant information retrieved from an extensive corpus, yet
encounters challenges when confronted with real-world noisy data. One recent
solution is to train a filter module to find relevant content but only achieve
suboptimal noise compression. In this paper, we propose to introduce the
information bottleneck theory into retrieval-augmented generation. Our approach
involves the filtration of noise by simultaneously maximizing the mutual
information between compression and ground output, while minimizing the mutual
information between compression and retrieved passage. In addition, we derive
the formula of information bottleneck to facilitate its application in novel
comprehensive evaluations, the selection of supervised fine-tuning data, and
the construction of reinforcement learning rewards. Experimental results
demonstrate that our approach achieves significant improvements across various
question answering datasets, not only in terms of the correctness of answer
generation but also in the conciseness with $2.5\%$ compression rate.

检索增强生成结合了大语言模型的能力和从广泛语料库检索到的相关信息，但在面对现实世界的噪声数据时面临挑战。本文提出将信息瓶颈理论引入检索增强生成，通过同时最大化压缩和生成结果的互信息，最小化压缩和检索到的内容之间的互信息，实现对噪声的过滤。实验结果表明，我们的方法在各种问答数据集中取得显著的改进，不仅在答案生成的正确性方面，而且在 2.5% 的压缩率下的简洁性方面也有明显提升。