Transformer-based models typically have a predefined bound to their input
length, because of their need to potentially attend to every token in the
input. In this work, we propose Unlimiformer: a general approach that can wrap
any existing pretrained encoder-decoder transformer, and offload the attention
computation across all layers to a single $k$-nearest-neighbor index; this
index can be kept on either the GPU or CPU memory and queried in sub-linear
time. This way, we can index extremely long input sequences, while every
attention head in every decoder layer retrieves its top-$k$ keys, instead of
attending to every key. We demonstrate Unlimiformers's efficacy on several
long-document and multi-document summarization benchmarks, showing that it can
summarize even 350k token-long inputs from the BookSum dataset, without any
input truncation at test time. Unlimiformer improves pretrained models such as
BART and Longformer by extending them to unlimited inputs without additional
learned weights and without modifying their code. We make our code and models
publicly available at this https URL .

这篇论文提出了一种名为 Unlimiformer 的新方法，可以在不需要修改模型代码或增加学习的权重的情况下，将具有谷歌的长文本模型 Longformer 和 Facebook 的 BART 等预训练模型拓展到可输入无限长度的输入序列，并在多篇文档摘要任务上表现出很高的效果。