Network Pruning is a promising way to address the huge computing resource
demands of the deployment and inference of Large Language Models (LLMs).
Retraining-free is important for LLMs' pruning methods. However, almost all of
the existing retraining-free pruning approaches for LLMs focus on unstructured
pruning, which requires specific hardware support for acceleration. In this
paper, we propose a novel retraining-free structured pruning framework for
LLMs, named FLAP (FLuctuation-based Adaptive Structured Pruning). It is
hardware-friendly by effectively reducing storage and enhancing inference
speed. For effective structured pruning of LLMs, we highlight three critical
elements that demand the utmost attention: formulating structured importance
metrics, adaptively searching the global compressed model, and implementing
compensation mechanisms to mitigate performance loss. First, FLAP determines
whether the output feature map is easily recoverable when a column of weight is
removed, based on the fluctuation pruning metric. Then it standardizes the
importance scores to adaptively determine the global compressed model
structure. At last, FLAP adds additional bias terms to recover the output
feature maps using the baseline values. We thoroughly evaluate our approach on
a variety of language benchmarks. Without any retraining, our method
significantly outperforms the state-of-the-art methods, including LLM-Pruner
and the extension of Wanda in structured pruning. The code is released at
this https URL

通过提出一种新颖的名为 FLAP（基于波动的自适应结构修剪）的网络学习模型无需再训练就能进行结构修剪的框架，可以有效减少存储和提高推理速度，大大优于现有的基于结构修剪的方法，同时通过制定结构重要性度量，自适应搜索全局压缩模型，并实施补偿机制来缓解性能损失。