Given the generational gap in available hardware between lay practitioners
and the most endowed institutions, LLMs are becoming increasingly inaccessible
as they grow in size. Whilst many approaches have been proposed to compress
LLMs to make their resource consumption manageable, these methods themselves
tend to be resource intensive, putting them out of the reach of the very user
groups they target. In this work, we explore the problem of structured pruning
of LLMs using only forward passes. We seek to empower practitioners to prune
models so large that their available hardware has just enough memory to run
inference. We develop Bonsai, a gradient-free, perturbative pruning method
capable of delivering small, fast, and accurate pruned models.
We observe that Bonsai outputs pruned models that (i) outperform those
generated by more expensive gradient-based structured pruning methods, and (ii)
are twice as fast (with comparable accuracy) as those generated by
semi-structured pruning methods requiring comparable resources as Bonsai. We
also leverage Bonsai to produce a new sub-2B model using a single A6000 that
yields state-of-the-art performance on 4/6 tasks on the Huggingface Open LLM
leaderboard.

利用仅使用前向传递的方法进行结构裁剪，我们开发了 Bonsai，一种无梯度、扰动性的裁剪方法，能够生成小型、快速、准确的模型，超越梯度裁剪方法和半结构裁剪方法的性能且节约资源。