Large pre-trained language models have demonstrated their proficiency in
storing factual knowledge within their parameters and achieving remarkable
results when fine-tuned for downstream natural language processing tasks.
Nonetheless, their capacity to access and manipulate knowledge with precision
remains constrained, resulting in performance disparities on
knowledge-intensive tasks when compared to task-specific architectures.
Additionally, the challenges of providing provenance for model decisions and
maintaining up-to-date world knowledge persist as open research frontiers. To
address these limitations, the integration of pre-trained models with
differentiable access mechanisms to explicit non-parametric memory emerges as a
promising solution. This survey delves into the realm of language models (LMs)
augmented with the ability to tap into external knowledge sources, including
external knowledge bases and search engines. While adhering to the standard
objective of predicting missing tokens, these augmented LMs leverage diverse,
possibly non-parametric external modules to augment their contextual processing
capabilities, departing from the conventional language modeling paradigm.
Through an exploration of current advancements in augmenting large language
models with knowledge, this work concludes that this emerging research
direction holds the potential to address prevalent issues in traditional LMs,
such as hallucinations, un-grounded responses, and scalability challenges.

将预训练语言模型与外部知识源相结合，可以增强其上下文处理能力，克服传统语言模型的问题，如错觉、无依据的回答和可拓展性挑战。