We present STAT: a simple algorithm to prune transformer models without any
fine-tuning. STAT eliminates both attention heads and neurons from the network,
while preserving accuracy by calculating a correction to the weights of the
next layer. Each layer block in the network is compressed using a series of
principled matrix factorizations that preserve the network structure. Our
entire algorithm takes minutes to compress BERT, and less than three hours to
compress models with 7B parameters using a single GPU. Using only several
hundred data examples, STAT preserves the output of the network and improves
upon existing gradient-free pruning methods. It is even competitive with
methods that include significant fine-tuning. We demonstrate our method on both
encoder and decoder architectures, including BERT, DistilBERT, and Llama-2
using benchmarks such as GLUE, Squad, WikiText2.

我们提出了 STAT 算法，一种简单的算法用于修剪 transformer 模型，而无需进行任何微调。STAT 通过计算下一层权重的修正来压缩网络中的注意力头和神经元，同时保持精度，并使用一系列有原则的矩阵分解方法来压缩网络结构。该算法仅需要几分钟来压缩 BERT，并且在单个 GPU 上使用不到三个小时来压缩具有 7B 参数的模型。STAT 仅使用几百个数据示例即可保留网络输出并改进现有的无梯度修剪方法，甚至能与包括重要微调的方法竞争。我们在编码器和解码器体系结构上展示了该方法，包括 BERT，DistilBERT 和 Llama-2，并使用 GLUE，Squad 和 WikiText2 等基准测试。