Large language models (LLMs) have revolutionized AI, but are constrained by
limited context windows, hindering their utility in tasks like extended
conversations and document analysis. To enable using context beyond limited
context windows, we propose virtual context management, a technique drawing
inspiration from hierarchical memory systems in traditional operating systems
that provide the appearance of large memory resources through data movement
between fast and slow memory. Using this technique, we introduce MemGPT
(Memory-GPT), a system that intelligently manages different memory tiers in
order to effectively provide extended context within the LLM's limited context
window, and utilizes interrupts to manage control flow between itself and the
user. We evaluate our OS-inspired design in two domains where the limited
context windows of modern LLMs severely handicaps their performance: document
analysis, where MemGPT is able to analyze large documents that far exceed the
underlying LLM's context window, and multi-session chat, where MemGPT can
create conversational agents that remember, reflect, and evolve dynamically
through long-term interactions with their users. We release MemGPT code and
data for our experiments at this https URL

通过虚拟上下文管理技术，提出了 MemGPT（Memory-GPT）系统，该系统能够在有限的上下文窗口中有效地提供扩展的上下文，从而在文档分析和多会话聊天等任务中克服现代大型语言模型的限制。