Structured pruning can simplify network architecture and improve inference
speed. Combined with the underlying hardware and inference engine in which the
final model is deployed, better results can be obtained by using latency
collaborative loss function to guide network pruning together. Existing pruning
methods that optimize latency have demonstrated leading performance, however,
they often overlook the hardware features and connection in the network. To
address this problem, we propose a global importance score SP-LAMP(Structured
Pruning Layer-Adaptive Magnitude-based Pruning) by deriving a global importance
score LAMP from unstructured pruning to structured pruning. In SP-LAMP, each
layer includes a filter with an SP-LAMP score of 1, and the remaining filters
are grouped. We utilize a group knapsack solver to maximize the SP-LAMP score
under latency constraints. In addition, we improve the strategy of collect the
latency to make it more accurate. In particular, for ResNet50/ResNet18 on
ImageNet and CIFAR10, SP-LAMP is 1.28x/8.45x faster with +1.7%/-1.57% top-1
accuracy changed, respectively. Experimental results in ResNet56 on CIFAR10
demonstrate that our algorithm achieves lower latency compared to alternative
approaches while ensuring accuracy and FLOPs.

通过使用延迟协同损失函数来指导网络修剪，我们提出了一个全局重要性评分 SP-LAMP，并利用组背包求解器在延迟约束下最大化 SP-LAMP 得分，取得了优于传统方法的结果，实验结果表明，我们的算法在确保准确性和 FLOPs 的情况下，达到了比替代方法更低的延迟。