Long-context large language models (LLMs) inference is increasingly critical, motivating a number of studies devoted to alleviating the substantial storage and computational costs in such scenarios. Layer-wise skipping methods are promising optimizations but rarely explored in long-context inference. We observe that existing layer-wise skipping strategies have several limitations when applied in long-context inference, including the inability to adapt to model and context variability, disregard for sublayer significance, and inapplicability for the prefilling phase. This paper proposes \sysname, an adaptive sublayer skipping method specifically designed for long-context inference. \sysname adaptively identifies less important layers by leveraging on-the-fly similarity information, enables sublayer-wise skipping, and accelerates both the prefilling and decoding phases. The effectiveness of \sysname is demonstrated through extensive experiments on various long-context benchmarks and models, showcasing its superior inference performance over existing baselines.

该研究解决了长上下文大型语言模型推理中的存储和计算成本问题。作者提出了一种名为AdaSkip的自适应子层跳过方法，利用即时相似性信息识别不重要的层，实现了子层跳过，加速了预填充和解码阶段。实验证明，AdaSkip在各类长上下文基准和模型上展现出优越的推理性能。

AdaSkip：加速长上下文大型语言模型推理的自适应子层跳过