Traditional pruning methods are known to be challenging to work in Large
Language Models (LLMs) for Generative AI because of their unaffordable training
process and large computational demands. For the first time, we introduce the
information entropy of hidden state features into a pruning metric design,
namely E-Sparse, to improve the accuracy of N:M sparsity on LLM. E-Sparse
employs the information richness to leverage the channel importance, and
further incorporates several novel techniques to put it into effect: (1) it
introduces information entropy to enhance the significance of parameter weights
and input feature norms as a novel pruning metric, and performs N:M sparsity
without modifying the remaining weights. (2) it designs global naive shuffle
and local block shuffle to quickly optimize the information distribution and
adequately cope with the impact of N:M sparsity on LLMs' accuracy. E-Sparse is
implemented as a Sparse-GEMM on FasterTransformer and runs on NVIDIA Ampere
GPUs. Extensive experiments on the LLaMA family and OPT models show that
E-Sparse can significantly speed up the model inference over the dense model
(up to 1.53X) and obtain significant memory saving (up to 43.52%), with
acceptable accuracy loss.

传统修剪方法在大型语言模型中的使用具有挑战性，因为训练过程代价高且计算需求大。我们首次引入隐藏状态特征的信息熵作为修剪度量设计，即 E-Sparse，以提高大型语言模型的 N:M 稀疏性准确性，并通过引入信息熵和几种创新技术来快速优化信息分布和应对 N:M 稀疏性对准确性的影响。E-Sparse 通过 FasterTransformer 实现为 Sparse-GEMM，并在 NVIDIA Ampere GPU 上运行。对 LLaMA 系列和 OPT 模型进行的广泛实验表明，E-Sparse 可以显著加速模型推理（高达 1.53 倍），并实现显著的内存节省（高达 43.52%），在可接受的准确性损失范围内。