Various Large Language Models(LLMs) from the Generative Pretrained
Transformer~(GPT) family have achieved outstanding performances in a wide range
of text generation tasks. However, the enormous model sizes have hindered their
practical use in real-world applications due to high inference latency.
Therefore, improving the efficiencies of LLMs through quantization, pruning,
and other means has been a key issue in LLM studies. In this work, we propose a
method based on Hessian sensitivity-aware mixed sparsity pruning to prune LLMs
to at least 50\% sparsity without the need of any retraining. It allocates
sparsity adaptively based on sensitivity, allowing us to reduce pruning-induced
error while maintaining the overall sparsity level. The advantages of the
proposed method exhibit even more when the sparsity is extremely high.
Furthermore, our method is compatible with quantization, enabling further
compression of LLMs.

通过基于 Hessian 灵敏度感知的混合稀疏剪枝方法，我们提出了一种剪枝 LLMs 的方法，以至少达到 50% 的稀疏度，而不需要任何重新训练，该方法适应性地分配稀疏度，减少了剪枝引起的错误，同时保持了整体稀疏度水平，并且在稀疏度极高时表现出更显著的优势，此外，我们的方法与量化兼容，从而进一步压缩 LLMs。