Recent work has aimed to improve LLM generations by filtering out
hallucinations, thereby improving the precision of the information in
responses. Correctness of a long-form response, however, also depends on the
recall of multiple pieces of information relevant to the question. In this
paper, we introduce Atomic Self-Consistency (ASC), a technique for improving
the recall of relevant information in an LLM response. ASC follows recent work,
Universal Self-Consistency (USC) in using multiple stochastic samples from an
LLM to improve the long-form response. Unlike USC which only focuses on
selecting the best single generation, ASC picks authentic subparts from the
samples and merges them into a superior composite answer. Through extensive
experiments and ablations, we show that merging relevant subparts of multiple
samples performs significantly better than picking a single sample. ASC
demonstrates significant gains over USC on multiple factoids and open-ended QA
datasets - ASQA, QAMPARI, QUEST, ELI5 with ChatGPT and Llama2. Our analysis
also reveals untapped potential for enhancing long-form generations using
approach of merging multiple samples.

提出了原子自洽（ASC）技术，通过合并多个样本中的相关子部分以提高长篇回答的信息检索能力，而不仅仅关注单个样本的抽取，该技术表现出明显优于 “Universal Self-Consistency”（USC）的效果，并在多个事实和开放式问答数据集上显示出未开发的提升长篇生成的潜力。

原子自洽的长篇生成改进

Atomic Self-Consistency for Better Long Form Generations

While recent language models have the ability to take long contexts as input,
relatively little is known about how well the language models use longer
context. We analyze language model performance on two tasks that require
identifying relevant information within their input contexts: multi-document
question answering and key-value retrieval. We find that performance is often
highest when relevant information occurs at the beginning or end of the input
context, and significantly degrades when models must access relevant
information in the middle of long contexts. Furthermore, performance
substantially decreases as the input context grows longer, even for explicitly
long-context models. Our analysis provides a better understanding of how
language models use their input context and provides new evaluation protocols
for future long-context models.

我们分析了语言模型在多文档问答和键值检索两个任务中的性能，发现当相关信息出现在输入上下文的开头或结尾时，性能通常最高，而在长上下文中访问相关信息时性能显著下降。此外，随着输入上下文的增长，性能显著降低，即使是专门用于长上下文的模型也如此。我们的分析对语言模型如何利用输入上下文提供了更好的理解，并为未来的长上下文模型提供了新的评估协议。

迷失在中间：语言模型如何使用长上下文

Lost in the Middle: How Language Models Use Long Contexts

Query expansion aims to mitigate the mismatch between the language used in a
query and in a document. However, query expansion methods can suffer from
introducing non-relevant information when expanding the query. To bridge this
gap, inspired by recent advances in applying contextualized models like BERT to
the document retrieval task, this paper proposes a novel query expansion model
that leverages the strength of the BERT model to select relevant document
chunks for expansion. In evaluation on the standard TREC Robust04 and GOV2 test
collections, the proposed BERT-QE model significantly outperforms BERT-Large
models.

本论文提供了一种使用 BERT 模型选择相关文档块进行扩展的新型查询扩展模型，能够在标准 TREC Robust04 测试集上显著优于 BERT-Large 模型，有助于解决查询扩展方法引入不相关信息的问题。