While current large language models (LLMs) demonstrate some capabilities in
knowledge-intensive tasks, they are limited by relying on their parameters as
an implicit storage mechanism. As a result, they struggle with infrequent
knowledge and temporal degradation. In addition, the uninterpretable nature of
parametric memorization makes it challenging to understand and prevent
hallucination. Parametric memory pools and model editing are only partial
solutions. Retrieval Augmented Generation (RAG) $\unicode{x2013}$ though
non-parametric $\unicode{x2013}$ has its own limitations: it lacks structure,
complicates interpretability and makes it hard to effectively manage stored
knowledge. In this paper, we introduce MemLLM, a novel method of enhancing LLMs
by integrating a structured and explicit read-and-write memory module. MemLLM
tackles the aforementioned challenges by enabling dynamic interaction with the
memory and improving the LLM's capabilities in using stored knowledge. Our
experiments indicate that MemLLM enhances the LLM's performance and
interpretability, in language modeling in general and knowledge-intensive tasks
in particular. We see MemLLM as an important step towards making LLMs more
grounded and factual through memory augmentation.

我们介绍了 MemLLM，这是一种通过整合结构化且显式的读写内存模块来增强 LLMs 的新方法。 MemLLM 通过使内存与 LLM 的动态交互，改善了 LLM 在使用存储的知识方面的能力，从而解决了上述挑战。我们的实验结果表明，MemLLM 提高了 LLM 的性能和可解释性，尤其是在语言建模和知识密集型任务中。我们认为 MemLLM 是使 LLMs 通过内存增强更加扎实和事实准确的重要一步。