The training and inference of large language models (LLMs) are together a
costly process that transports knowledge from raw data to meaningful
computation. Inspired by the memory hierarchy of the human brain, we reduce
this cost by equipping LLMs with explicit memory, a memory format cheaper than
model parameters and text retrieval-augmented generation (RAG). Conceptually,
with most of its knowledge externalized to explicit memories, the LLM can enjoy
a smaller parameter size, training cost, and inference cost, all proportional
to the amount of remaining "abstract knowledge". As a preliminary proof of
concept, we train from scratch a 2.4B LLM, which achieves better performance
than much larger LLMs as well as RAG models, and maintains higher decoding
speed than RAG. The model is named $\text{Memory}^3$, since explicit memory is
the third form of memory in LLMs after implicit memory (model parameters) and
working memory (context key-values). We introduce a memory circuitry theory to
support the externalization of knowledge, and present novel techniques
including a memory sparsification mechanism that makes storage tractable and a
two-stage pretraining scheme that facilitates memory formation.

利用显式记忆将大型语言模型（LLMs）的训练和推理成本降低，实现更小的参数大小、训练成本和推理成本，从而传递意义的计算。