Overparametrized transformer networks are the state-of-the-art architecture
for Large Language Models (LLMs). However, such models contain billions of
parameters making large compute a necessity, while raising environmental
concerns. To address these issues, we propose FinerCut, a new form of
fine-grained layer pruning, which in contrast to prior work at the transformer
block level, considers all self-attention and feed-forward network (FFN) layers
within blocks as individual pruning candidates. FinerCut prunes layers whose
removal causes minimal alternation to the model's output -- contributing to a
new, lean, interpretable, and task-agnostic pruning method. Tested across 9
benchmarks, our approach retains 90% performance of Llama3-8B with 25% layers
removed, and 95% performance of Llama3-70B with 30% layers removed, all without
fine-tuning or post-pruning reconstruction. Strikingly, we observe intriguing
results with FinerCut: 42% (34 out of 80) of the self-attention layers in
Llama3-70B can be removed while preserving 99% of its performance -- without
additional fine-tuning after removal. Moreover, FinerCut provides a tool to
inspect the types and locations of pruned layers, allowing to observe
interesting pruning behaviors. For instance, we observe a preference for
pruning self-attention layers, often at deeper consecutive decoder layers. We
hope our insights inspire future efficient LLM architecture designs.

基于超参数的 Transformer 网络是大规模语言模型的最新架构。本文提出了一种名为 FinerCut 的剪枝方法，它能够剪枝 Transformer 网络中的自注意力层和前馈神经网络层，以达到模型精简、可解释、多任务的剪枝效果。与之前的工作相比，FinerCut 的效果优于大多数任务，无需微调或后剪枝重建。该方法还提供了可视化工具，以观察被剪枝的层的类型和位置，为未来高效的语言模型设计提供了灵感。