Large language models (LLMs) have proven to be highly effective across
various natural language processing tasks. However, their large number of
parameters poses significant challenges for practical deployment. Pruning, a
technique aimed at reducing the size and complexity of LLMs, offers a potential
solution by removing redundant components from the network. Despite the promise
of pruning, existing methods often struggle to achieve substantial end-to-end
LLM inference speedup. In this paper, we introduce SLEB, a novel approach
designed to streamline LLMs by eliminating redundant transformer blocks. We
choose the transformer block as the fundamental unit for pruning, because LLMs
exhibit block-level redundancy with high similarity between the outputs of
neighboring blocks. This choice allows us to effectively enhance the processing
speed of LLMs. Our experimental results demonstrate that SLEB successfully
accelerates LLM inference without compromising the linguistic capabilities of
these models, making it a promising technique for optimizing the efficiency of
LLMs. The code is available at: this https URL

利用 SLEB 方法，通过消除多余的 transformer 块来优化大型语言模型的效率，加速模型的推断过程，同时不损害模型的语言能力。