To process contexts with unlimited length using Large Language Models (LLMs),
recent studies explore hierarchically managing the long text. Only several text
fragments are taken from the external memory and passed into the temporary
working memory, i.e., LLM's context window. However, existing approaches
isolatedly handle the text fragments without considering their structural
connections, thereby suffering limited capability on texts with intensive
inter-relations, e.g., coherent stories and code repositories. This work
attempts to resolve this by exploiting the fragment-level relations in external
memory. First, we formulate the fragment-level relations and present several
instantiations for different text types. Next, we introduce a relation-aware
fragment assessment criteria upon previous independent fragment assessment.
Finally, we present the fragment-connected Hierarchical Memory based LLM. We
validate the benefits of involving these relations on long story understanding,
repository-level code generation, and long-term chatting.

利用片段级别关系来处理长文本，通过引入基于关系的片段评估标准以及基于片段连接的分层记忆的大型语言模型，提高了长篇故事理解、代码生成和长期对话的性能。

FragRel: 大型语言模型外部存储中利用片段级关系

FragRel: Exploiting Fragment-level Relations in the External Memory of  Large Language Models

Fine-tuning pre-trained language models (PLMs) achieves impressive
performance on a range of downstream tasks, and their sizes have consequently
been getting bigger. Since a different copy of the model is required for each
task, this paradigm is infeasible for storage-constrained edge devices like
mobile phones. In this paper, we propose SPARTAN, a parameter efficient (PE)
and computationally fast architecture for edge devices that adds hierarchically
organized sparse memory after each Transformer layer. SPARTAN freezes the PLM
parameters and fine-tunes only its memory, thus significantly reducing storage
costs by re-using the PLM backbone for different tasks. SPARTAN contains two
levels of memory, with only a sparse subset of parents being chosen in the
first level for each input, and children cells corresponding to those parents
being used to compute an output representation. This sparsity combined with
other architecture optimizations improves SPARTAN's throughput by over 90%
during inference on a Raspberry Pi 4 when compared to PE baselines (adapters)
while also outperforming the latter by 0.1 points on the GLUE benchmark.
Further, it can be trained 34% faster in a few-shot setting, while performing
within 0.9 points of adapters. Qualitative analysis shows that different parent
cells in SPARTAN specialize in different topics, thus dividing responsibility
efficiently.

本研究提出了一种基于 SPARTAN 的架构，在使用预训练语言模型的基础上，仅对其记忆进行微调，以显着减少存储和计算成本，从而适用于存储受限的边缘设备，并通过使用分层组织的稀疏内存以及其他架构优化，在不损失性能的情况下提高了其执行效率。

SPARTAN：用于参数高效的 Transformer 的稀疏分层存储器

SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers

Neural memory enables fast adaptation to new tasks with just a few training
samples. Existing memory models store features only from the single last layer,
which does not generalize well in presence of a domain shift between training
and test distributions. Rather than relying on a flat memory, we propose a
hierarchical alternative that stores features at different semantic levels. We
introduce a hierarchical prototype model, where each level of the prototype
fetches corresponding information from the hierarchical memory. The model is
endowed with the ability to flexibly rely on features at different semantic
levels if the domain shift circumstances so demand. We meta-learn the model by
a newly derived hierarchical variational inference framework, where
hierarchical memory and prototypes are jointly optimized. To explore and
exploit the importance of different semantic levels, we further propose to
learn the weights associated with the prototype at each level in a data-driven
way, which enables the model to adaptively choose the most generalizable
features. We conduct thorough ablation studies to demonstrate the effectiveness
of each component in our model. The new state-of-the-art performance on
cross-domain and competitive performance on traditional few-shot classification
further substantiates the benefit of hierarchical variational memory.

本研究提出了一种基于分层原型模型和层次内存的神经记忆方法，该方法能够在领域转移的情况下，灵活地依赖不同语义层面的特征来适应新任务，而数据驱动的学习方法进一步可以使模型自适应地选择最具通用性的特征。该方法通过大量消融实验验证了性能的显著提升，尤其在跨领域和少样本分类方面表现出了显著优势。