Large Language Models (LLMs) exhibit substantial capabilities yet encounter
challenges, including hallucination, outdated knowledge, and untraceable
reasoning processes. Retrieval-augmented generation (RAG) has emerged as a
promising solution, integrating knowledge from external databases to mitigate
these challenges. However, inappropriate retrieved passages can potentially
hinder the LLMs' capacity to generate comprehensive and high-quality responses.
Prior RAG studies on the robustness of retrieval noises often confine
themselves to a limited set of noise types, deviating from real-world retrieval
environments and limiting practical applicability. In this study, we initially
investigate retrieval noises and categorize them into three distinct types,
reflecting real-world environments. We analyze the impact of these various
retrieval noises on the robustness of LLMs. Subsequently, we propose a novel
RAG approach known as Retrieval-augmented Adaptive Adversarial Training (RAAT).
RAAT leverages adaptive adversarial training to dynamically adjust the model's
training process in response to retrieval noises. Concurrently, it employs
multi-task learning to ensure the model's capacity to internally recognize
noisy contexts. Extensive experiments demonstrate that the LLaMA-2 7B model
trained using RAAT exhibits significant improvements in F1 and EM scores under
diverse noise conditions. For reproducibility, we release our code and data at:
this https URL

这项研究将大型语言模型与检索增强生成相结合，提出了一种名为 RAAT 的新的检索增强自适应对抗训练方法，通过适应性对抗训练动态调整模型的训练过程以应对检索噪声，并通过多任务学习确保模型内部识别噪声上下文的能力。实验证明，使用 RAAT 训练的 LLaMA-2 7B 模型在不同噪声条件下显著提高了 F1 和 EM 分数。