BriefGPT.xyz
Jun, 2024
SinkLoRA:增强效率与聊天能力的大型长文本语言模型
SinkLoRA: Enhanced Efficiency and Chat Capabilities for Long-Context Large Language Models
HTML
PDF
Hengyu Zhang
TL;DR
扩展Transformer模型以适应更长的序列长度是一项关键挑战,本文提出了SinkLoRA作为应对策略,通过改进工作分配和应用高效的缓存压缩算法来提高模型性能。
Abstract
Extending the functionality of the
transformer model
to accommodate longer
sequence lengths
has become a critical challenge. This extension is crucial not only for improving tasks such as language translation and
→