This paper introduces channel gating, a dynamic, fine-grained, and
hardware-efficient pruning scheme to reduce the computation cost for
convolutional neural networks (CNNs). Channel gating identifies regions in the
features that contribute less to the classification result, and skips the
computation on a subset of the input channels for these ineffective regions.
Unlike static network pruning, channel gating optimizes CNN inference at
run-time by exploiting input-specific characteristics, which allows
substantially reducing the compute cost with almost no accuracy loss. We
experimentally show that applying channel gating in state-of-the-art networks
achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and
2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy
loss on CIFAR-10. Combining our method with knowledge distillation reduces the
compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We
further demonstrate that channel gating can be realized in hardware
efficiently. Our approach exhibits sparsity patterns that are well-suited to
dense systolic arrays with minimal additional hardware. We have designed an
accelerator for channel gating networks, which can be implemented using either
FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our
accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a
theoretical FLOP reduction of 2.8$\times$.

本研究介绍了通道门控（channel gating）方法，该方法是动态、细粒度且硬件高效的裁剪方案，能够通过跳过对分类结果没有贡献的输入通道的计算，优化卷积神经网络。实验证明，该方法能够在几乎不损失准确度的情况下，实现浮点运算量减少 2.7-8.0 倍，内存访问减少 2.0-4.4 倍，并结合知识蒸馏可以进一步降低计算成本。我们还设计了一个加速器，能够以 2.4 倍的速度进行量化的 ResNet-18 模型的推理，并实现了 2.8 倍的理论 FLOP 减少。