BriefGPT.xyz
Jan, 2024
从4K到400K:用激活信标扩展LLM的上下文
Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon
HTML
PDF
Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye...
TL;DR
利用Activation Beacon插件来压缩语言模型的原始激活,从而使其在有限上下文窗口的情况下能感知更长的上下文,提高LLM的长文本处理能力。
Abstract
The utilization of
long contexts
poses a big challenge for large
language models
due to their limited
context window
length. Although the
→