Retrieval-augmented generation integrates the capabilities of large language
models with relevant information retrieved from an extensive corpus, yet
encounters challenges when confronted with real-world noisy data. One recent
solution is to train a filter module to find relevant content but only achieve
suboptimal noise compression. In this paper, we propose to introduce the
information bottleneck theory into retrieval-augmented generation. Our approach
involves the filtration of noise by simultaneously maximizing the mutual
information between compression and ground output, while minimizing the mutual
information between compression and retrieved passage. In addition, we derive
the formula of information bottleneck to facilitate its application in novel
comprehensive evaluations, the selection of supervised fine-tuning data, and
the construction of reinforcement learning rewards. Experimental results
demonstrate that our approach achieves significant improvements across various
question answering datasets, not only in terms of the correctness of answer
generation but also in the conciseness with $2.5\%$ compression rate.

检索增强生成结合了大语言模型的能力和从广泛语料库检索到的相关信息，但在面对现实世界的噪声数据时面临挑战。本文提出将信息瓶颈理论引入检索增强生成，通过同时最大化压缩和生成结果的互信息，最小化压缩和检索到的内容之间的互信息，实现对噪声的过滤。实验结果表明，我们的方法在各种问答数据集中取得显著的改进，不仅在答案生成的正确性方面，而且在 2.5% 的压缩率下的简洁性方面也有明显提升。

检索增强生成中有效的噪音过滤的信息瓶颈视角

An Information Bottleneck Perspective for Effective Noise Filtering on  Retrieval-Augmented Generation

Reasoning about time is of fundamental importance. Many facts are
time-dependent. For example, athletes change teams from time to time, and
different government officials are elected periodically. Previous
time-dependent question answering (QA) datasets tend to be biased in either
their coverage of time spans or question types. In this paper, we introduce a
comprehensive probing dataset \tempreason to evaluate the temporal reasoning
capability of large language models. Our dataset includes questions of three
temporal reasoning levels. In addition, we also propose a novel learning
framework to improve the temporal reasoning capability of large language
models, based on temporal span extraction and time-sensitive reinforcement
learning. We conducted experiments in closed book QA, open book QA, and
reasoning QA settings and demonstrated the effectiveness of our approach. Our
code and data are released on this https URL

本文介绍了一个全面的测试数据 	empreason 来评估大语言模型的时间推理能力，包括三个时间推理水平的问题，并提出了一种基于时间跨度提取和时间敏感的强化学习的新型学习框架来提高其时间推理能力，并证明了其有效性。