The retrieval-augmented generation (RAG) enables retrieval of relevant
information from an external knowledge source and allows large language models
(LLMs) to answer queries over previously unseen document collections. However,
it was demonstrated that traditional RAG applications perform poorly in
answering multi-hop questions, which require retrieving and reasoning over
multiple elements of supporting evidence. We introduce a new method called
Multi-Meta-RAG, which uses database filtering with LLM-extracted metadata to
improve the RAG selection of the relevant documents from various sources,
relevant to the question. While database filtering is specific to a set of
questions from a particular domain and format, we found out that Multi-Meta-RAG
greatly improves the results on the MultiHop-RAG benchmark. The code is
available at this https URL

通过使用数据库过滤和对 LLM 提取的元数据，Multi-Meta-RAG 方法改进了 RAG 在多跳问题上的表现，从而提高了从各种来源中选择相关文档的准确性。

多源元 RAG: 使用 LLM 提取的元数据进行数据库过滤以改进用于多跳查询的 RAG

Multi-Meta-RAG: Improving RAG for Multi-Hop Queries using Database  Filtering with LLM-Extracted Metadata

Unstructured Electronic Health Record (EHR) data often contains critical
information complementary to imaging data that would inform radiologists'
diagnoses. However, time constraints and the large volume of notes frequently
associated with individual patients renders manual perusal of such data to
identify relevant evidence infeasible in practice. Modern Large Language Models
(LLMs) provide a flexible means of interacting with unstructured EHR data, and
may provide a mechanism to efficiently retrieve and summarize unstructured
evidence relevant to a given query. In this work, we propose and evaluate an
LLM (Flan-T5 XXL) for this purpose. Specifically, in a zero-shot setting we
task the LLM to infer whether a patient has or is at risk of a particular
condition; if so, we prompt the model to summarize the supporting evidence.
Enlisting radiologists for manual evaluation, we find that this LLM-based
approach provides outputs consistently preferred to a standard information
retrieval baseline, but we also highlight the key outstanding challenge: LLMs
are prone to hallucinating evidence. However, we provide results indicating
that model confidence in outputs might indicate when LLMs are hallucinating,
potentially providing a means to address this.

使用大型语言模型（Flan-T5 XXL）从非结构化电子健康记录（EHR）数据中检索和总结与特定查询相关的论据，较传统信息检索方法更受放射科医生欢迎，但面临 LLMs 产生虚构证据的挑战。然而，模型的信心可能指示 LLMs 是否存在虚构，为解决此问题提供了潜在方法。

利用 LLMs 从电子病历中检索证据：可能性与挑战

Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges

Recent large language models often answer factual questions correctly. But
users can't trust any given claim a model makes without fact-checking, because
language models can hallucinate convincing nonsense. In this work we use
reinforcement learning from human preferences (RLHP) to train "open-book" QA
models that generate answers whilst also citing specific evidence for their
claims, which aids in the appraisal of correctness. Supporting evidence is
drawn from multiple documents found via a search engine, or from a single
user-provided document. Our 280 billion parameter model, GopherCite, is able to
produce answers with high quality supporting evidence and abstain from
answering when unsure. We measure the performance of GopherCite by conducting
human evaluation of answers to questions in a subset of the NaturalQuestions
and ELI5 datasets. The model's response is found to be high-quality 80\% of the
time on this Natural Questions subset, and 67\% of the time on the ELI5 subset.
Abstaining from the third of questions for which it is most unsure improves
performance to 90\% and 80\% respectively, approaching human baselines.
However, analysis on the adversarial TruthfulQA dataset shows why citation is
only one part of an overall strategy for safety and trustworthiness: not all
claims supported by evidence are true.

通过强化学习从人类偏好中得出支持性证据，训练生成回答并支持其声称的 “开放式书目” QA 模型。该模型能够从搜索引擎中找到的多个文档或单个用户提供的文档中提取支持证据。通过在 NaturalQuestions 和 ELI5 数据集的子集中进行的人类评估，该模型的响应在这两个子集中 80％和 67％的时间内达到高质量水平，但并非所有声称都被支持的证据是正确的。