Standard language models generate text by selecting tokens from a fixed,
finite, and standalone vocabulary. We introduce a novel method that selects
context-aware phrases from a collection of supporting documents. One of the
most significant challenges for this paradigm shift is determining the training
oracles, because a string of text can be segmented in various ways and each
segment can be retrieved from numerous possible documents. To address this, we
propose to initialize the training oracles using linguistic heuristics and,
more importantly, bootstrap the oracles through iterative self-reinforcement.
Extensive experiments show that our model not only outperforms standard
language models on a variety of knowledge-intensive tasks but also demonstrates
improved generation quality in open-ended text generation. For instance,
compared to the standard language model counterpart, our model raises the
accuracy from 23.47% to 36.27% on OpenbookQA, and improves the MAUVE score from
42.61% to 81.58% in open-ended text generation. Remarkably, our model also
achieves the best performance and the lowest latency among several
retrieval-augmented baselines. In conclusion, we assert that retrieval is more
accurate generation and hope that our work will encourage further research on
this new paradigm shift.

我们提出了一种新颖的方法，通过从一系列支持文件中选择上下文相关的短语来生成文本，并使用语言学启发式方法初始化训练阐述，并通过迭代的自我增强来加强训练阐述。实验证明，与标准语言模型相比，我们的模型在各种知识密集型任务上不仅表现出色，而且在开放式文本生成中表现出更高的生成质量。我们的模型在 OpenbookQA 上将准确性从 23.47% 提高到 36.27%，在开放式文本生成中的 MAUVE 得分从 42.61% 提高到 81.58%。值得注意的是，我们的模型在几个检索增强基准中实现了最佳性能和最低延迟。总之，我们断言检索是更准确的生成，并希望我们的工作能够鼓励进一步研究这一新的转变。