Retrieval-augmented language models (RALMs) hold promise to produce language
understanding systems that are are factual, efficient, and up-to-date. An
important desideratum of RALMs, is that retrieved information helps model
performance when it is relevant, and does not harm performance when it is not.
This is particularly important in multi-hop reasoning scenarios, where misuse
of irrelevant evidence can lead to cascading errors. However, recent work has
shown that retrieval augmentation can sometimes have a negative effect on
performance. In this work, we present a thorough analysis on five open-domain
question answering benchmarks, characterizing cases when retrieval reduces
accuracy. We then propose two methods to mitigate this issue. First, a simple
baseline that filters out retrieved passages that do not entail question-answer
pairs according to a natural language inference (NLI) model. This is effective
in preventing performance reduction, but at a cost of also discarding relevant
passages. Thus, we propose a method for automatically generating data to
fine-tune the language model to properly leverage retrieved passages, using a
mix of relevant and irrelevant contexts at training time. We empirically show
that even 1,000 examples suffice to train the model to be robust to irrelevant
contexts while maintaining high performance on examples with relevant ones.

使用检索增强语言模型 (Retrieval-augmented language models, RALMs) 在处理特定应用场景问题时，检索到的信息能够提高模型的性能，并且不会损害性能。该研究分析了五个开放领域问答基准，并提出了两种方法以减轻性能下降问题。

提升检索增强语言模型对无关背景的鲁棒性

Making Retrieval-Augmented Language Models Robust to Irrelevant Context

Natural language question answering over knowledge graphs is an important and
interesting task as it enables common users to gain accurate answers in an easy
and intuitive manner. However, it remains a challenge to bridge the gap between
unstructured questions and structured knowledge graphs. To address the problem,
a natural discipline is building a structured query to represent the input
question. Searching the structured query over the knowledge graph can produce
answers to the question. Distinct from the existing methods that are based on
semantic parsing or templates, we propose an effective approach powered by a
novel notion, structural query pattern, in this paper. Given an input question,
we first generate its query sketch that is compatible with the underlying
structure of the knowledge graph. Then, we complete the query graph by labeling
the nodes and edges under the guidance of the structural query pattern.
Finally, answers can be retrieved by executing the constructed query graph over
the knowledge graph. Evaluations on three question answering benchmarks show
that our proposed approach outperforms state-of-the-art methods significantly.

提出了一种有效的自然结构查询模式方法，该方法可实现基于知识图谱的自然语言问答并在三个问答基准测试中显著优于现有方法。