Large language models (LLMs) have demonstrated the world with the sparks of
artificial general intelligence (AGI). One opinion, especially from some
startups working on LLMs, argues that an LLM with nearly unlimited context
length can realize AGI. However, they might be too optimistic about the
long-context capability of (existing) LLMs -- (1) Recent literature has shown
that their effective context length is significantly smaller than their claimed
context length; and (2) Our reasoning-in-a-haystack experiments further
demonstrate that simultaneously finding the relevant information from a long
context and conducting (simple) reasoning is nearly impossible. In this paper,
we envision a pathway from LLMs to AGI through the integration of
\emph{memory}. We believe that AGI should be a system where LLMs serve as core
processors. In addition to raw data, the memory in this system would store a
large number of important conclusions derived from reasoning processes.
Compared with retrieval-augmented generation (RAG) that merely processing raw
data, this approach not only connects semantically related information closer,
but also simplifies complex inferences at the time of querying. As an
intermediate stage, the memory will likely be in the form of natural language
descriptions, which can be directly consumed by users too. Ultimately, every
agent/person should have its own large personal model, a deep neural network
model (thus \emph{AI-native}) that parameterizes and compresses all types of
memory, even the ones cannot be described by natural languages. Finally, we
discuss the significant potential of AI-native memory as the transformative
infrastructure for (proactive) engagement, personalization, distribution, and
social in the AGI era, as well as the incurred privacy and security challenges
with preliminary solutions.

通过集成记忆，我们设想了从大型语言模型（LLMs）到人工通用智能（AGI）的一种路径。我们认为 AGI 应该是一个 LLM 作为核心处理器的系统，其中除了原始数据外，该系统的记忆还可以存储推理过程中得出的大量重要结论。相比仅处理原始数据的检索增强生成（RAG）方法，这种方法不仅更紧密地连接语义相关的信息，还可以在查询时简化复杂的推理。最终，每个个体 / 人都应该有自己的大型个人模型，即可以参数化和压缩所有类型记忆的深度神经网络模型（因此是 AI 本地的）。最后，我们讨论了 AI 本地记忆作为（主动的）参与、个性化、分发和社会在 AGI 时代的变革基础设施的巨大潜力，以及相应的隐私和安全挑战与初步解决方案。