As neural networks grow in size and complexity, inference speeds decline. To
combat this, one of the most effective compression techniques -- channel
pruning -- removes channels from weights. However, for multi-branch segments of
a model, channel removal can introduce inference-time memory copies. In turn,
these copies increase inference latency -- so much so that the pruned model can
be slower than the unpruned model. As a workaround, pruners conventionally
constrain certain channels to be pruned together. This fully eliminates memory
copies but, as we show, significantly impairs accuracy. We now have a dilemma:
Remove constraints but increase latency, or add constraints and impair
accuracy. In response, our insight is to reorder channels at export time, (1)
reducing latency by reducing memory copies and (2) improving accuracy by
removing constraints. Using this insight, we design a generic algorithm UPSCALE
to prune models with any pruning pattern. By removing constraints from existing
pruners, we improve ImageNet accuracy for post-training pruned models by 2.1
points on average -- benefiting DenseNet (+16.9), EfficientNetV2 (+7.9), and
ResNet (+6.2). Furthermore, by reordering channels, UPSCALE improves inference
speeds by up to 2x over a baseline export.

神经网络规模和复杂性增加时，推理速度下降。通道剪枝是一种最有效的压缩技术之一，它从权重中删除通道。然而，在模型的多分支部分，通道的删除可能会引入推理时的内存复制。为了解决这个问题，我们提出了一种新的算法 UPSCALE，通过重新排序通道以减少内存复制和去除约束来提高模型性能。该算法在后训练剪枝模型的 ImageNet 准确性上平均提高 2.1 个百分点，并且可提高高出基线导出的推理速度多达 2 倍。