In this paper, we demonstrate the benefits of using memory augmented Large
Language Model (LLM) architecture in improving the recall abilities of facts
from a potentially long context. As a case study we test LARIMAR, a recently
proposed LLM architecture which augments a LLM decoder with an external
associative memory, on several long-context recall tasks, including passkey and
needle-in-the-haystack tests. We demonstrate that the external memory can be
adapted at test time to handle contexts much longer than those seen during
training, while keeping readouts from the memory recognizable to the trained
decoder and without increasing GPU memory footprint. Compared to alternative
architectures for long-context recall tasks with models of a comparable
parameter count, LARIMAR is able to maintain strong performance without any
task-specific training.

本文展示了使用增强记忆的大型语言模型（LLM）架构在提高从潜在长上下文中召回事实的能力方面的好处。我们以 LARIMAR 为案例研究，它是最近提出的一种 LLM 架构，通过在 LLM 解码器上增加外部关联内存来增强性能，并在几个长上下文召回任务中进行测试，包括密码测试和大海捞针测试。我们证明了测试时可以适应比训练中观察到的更长上下文，同时保持经过训练的解码器可以识别的内存读出结果，而不增加 GPU 内存占用。与参数数量相近的长上下文召回任务的其他替代架构相比，LARIMAR 可以在没有任何特定任务训练的情况下保持强大的性能。

基于内存的大型语言模型中的针头引线

Needle in the Haystack for Memory Based Large Language Models

Efficient and accurate updating of knowledge stored in Large Language Models
(LLMs) is one of the most pressing research challenges today. This paper
presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with
a distributed episodic memory. Larimar's memory allows for dynamic, one-shot
updates of knowledge without the need for computationally expensive re-training
or fine-tuning. Experimental results on multiple fact editing benchmarks
demonstrate that Larimar attains accuracy comparable to most competitive
baselines, even in the challenging sequential editing setup, but also excels in
speed - yielding speed-ups of 4-10x depending on the base LLM - as well as
flexibility due to the proposed architecture being simple, LLM-agnostic, and
hence general. We further provide mechanisms for selective fact forgetting and
input context length generalization with Larimar and show their effectiveness.

Larimar 是一种新颖的、仿脑结构的体系架构，用于增强大型语言模型（LLMs）的分布式情景记忆，实现动态、一次性的知识更新，无需计算昂贵的重新训练或微调。实验结果表明，Larimar 在多个事实编辑基准上达到与竞争对手相当的准确率，甚至在挑战性的连续编辑设置中也表现出色，并且由于其简单、LLM 无关的架构，还具有速度快、灵活性高的优势。我们进一步提供了选择性事实遗忘和输入上下文长度泛化的机制，并展示了它们的有效性。