We present SNOWS, a one-shot post-training pruning framework aimed at reducing the cost of vision network inference without retraining. Current leading one-shot pruning methods minimize layer-wise least squares reconstruction error which does not take into account deeper network representations. We propose to optimize a more global reconstruction objective. This objective accounts for nonlinear activations deep in the network to obtain a better proxy for the network loss. This nonlinear objective leads to a more challenging optimization problem -- we demonstrate it can be solved efficiently using a specialized second-order optimization framework. A key innovation of our framework is the use of Hessian-free optimization to compute exact Newton descent steps without needing to compute or store the full Hessian matrix. A distinct advantage of SNOWS is that it can be readily applied on top of any sparse mask derived from prior methods, readjusting their weights to exploit nonlinearities in deep feature representations. SNOWS obtains state-of-the-art results on various one-shot pruning benchmarks including residual networks and Vision Transformers (ViT/B-16 and ViT/L-16, 86m and 304m parameters respectively).

本文提出了SNOWS，一种一次性后训练剪枝框架，旨在降低视觉网络推理的成本而无需重新训练。当前的方法未能充分考虑深层网络表示，作者提出了一个全局重构目标，并通过无海森优化高效地解决了这一挑战性优化问题，SNOWS在多项一次性剪枝基准测试中实现了最先进的结果。

一次性剪枝中保持深层表示：无海森二阶优化框架