Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce \textbf{EpMAN} -- a method for processing long contexts in an \textit{episodic memory} module while \textit{holistically attending to} semantically relevant context chunks. The output of \textit{episodic attention} is then used to reweigh the decoder's self-attention to the stored KV cache of the context during training and generation. When an LLM decoder is trained using \textbf{EpMAN}, its performance on multiple challenging single-hop long-context recall and question-answering benchmarks is found to be stronger and more robust across the range from 16k to 256k tokens than baseline decoders trained with self-attention, and popular retrieval-augmented generation frameworks.

本研究针对大型语言模型（LLMs）在处理长上下文时的效率问题，提出了一种新的方法EpMAN，通过情节记忆模块对语义相关的上下文块进行整体关注。实验结果表明，使用EpMAN训练的LLM解码器在多项具有挑战性的长上下文回忆和问答基准测试中表现出更强的韧性和优越性。

EpMAN：用于推广到更长上下文的情节记忆注意力