BriefGPT.xyz
Jul, 2024
弹性缓存的视觉指令遵循模型高效推理
Efficient Inference of Vision Instruction-Following Models with Elastic Cache
HTML
PDF
Zuyan Liu, Benlin Liu, Jiahui Wang, Yuhao Dong, Guangyi Chen...
TL;DR
本研究解决了多模态指令遵循模型在高内存需求下的缓存管理问题。提出的弹性缓存方法通过对指令编码和输出生成阶段采用不同的加速策略,创新性地引入重要性驱动的缓存合并策略,提升了上下文信息的保留。实验结果表明,弹性缓存显著提高了模型效率,并在多项语言生成任务中优于现有的修剪方法。
Abstract
In the field of instruction-following large
Vision-Language Models
(LVLMs), the efficient deployment of these models faces challenges, notably due to the high memory demands of their key-value (KV) caches. Conventional
→