The recent focus on the efficiency of deep neural networks (DNNs) has led to
significant work on model compression approaches, of which weight pruning is
one of the most popular. At the same time, there is rapidly-growing
computational support for efficiently executing the unstructured-sparse models
obtained via pruning. Yet, most existing pruning methods minimize just the
number of remaining weights, i.e. the size of the model, rather than optimizing
for inference time. We address this gap by introducing SPDY, a new compression
method which automatically determines layer-wise sparsity targets achieving a
desired inference speedup on a given system, while minimizing accuracy loss.
SPDY is composed of two new techniques: the first is an efficient dynamic
programming algorithm for solving the speedup-constrained layer-wise
compression problem assuming a set of given layer-wise sensitivity scores; the
second is a local search procedure for determining accurate layer-wise
sensitivity scores. Experiments across popular vision and language models show
that SPDY guarantees speedups while recovering higher accuracy relative to
existing strategies, both for one-shot and gradual pruning scenarios, and is
compatible with most existing pruning approaches. We also extend our approach
to the recently-proposed task of pruning with very little data, where we
achieve the best known accuracy recovery when pruning to the GPU-supported 2:4
sparsity pattern.

SPDY 是一种新的压缩方法，自动确定逐层稀疏度目标并最小化精度损失，从而实现所需的推理加速度，并且在流行的视觉和语言模型上表现良好。