BriefGPT.xyz
Jan, 2023
SparseGPT：一次修剪即可在大型语言模型上进行精准压缩
Massive Language Models Can Be Accurately Pruned in One-Shot
HTML
PDF
Elias Frantar, Dan Alistarh
TL;DR
本文提出了一种名为SparseGPT的新型剪枝方法，能够高效、准确地应用于海量的GPT模型，实现一次性稀疏化至少50％，并在几乎不影响困惑度的情况下，将最大可用的开源模型OPT-175B和BLOOM-176B稀疏化至60％。
Abstract
We show for the first time that large-scale generative pretrained transformer (
gpt
) family models can be pruned to at least 50%
sparsity
in one-shot, without any retraining, at minimal loss of accuracy. This is a
→