The Winograd or Cook-Toom class of algorithms help to reduce the overall
compute complexity of many modern deep convolutional neural networks (CNNs).
Although there has been a lot of research done on model and algorithmic
optimization of CNN, little attention has been paid to the efficient
implementation of these algorithms on embedded CPUs, which usually have very
limited memory and low power budget. This paper aims to fill this gap and
focuses on the efficient implementation of Winograd or Cook-Toom based
convolution on modern Arm Cortex-A CPUs, widely used in mobile devices today.
Specifically, we demonstrate a reduction in inference latency by using a set of
optimization strategies that improve the utilization of computational
resources, and by effectively leveraging the ARMv8-A NEON SIMD instruction set.
We evaluated our proposed region-wise multi-channel implementations on Arm
Cortex-A73 platform using several representative CNNs. The results show
significant performance improvements in full network, up to 60%, over existing
im2row/im2col based optimization techniques

本文针对移动设备上广泛使用的 ARM Cortex-A CPU，探讨了 Winograd 或 Cook-Toom 特征压缩算法在卷积神经网络上的高效实现，通过优化计算资源的利用和充分发挥 ARMv8-A NEON SIMD 指令集等策略降低了推断延迟，并在数个代表性 CNN 上进行了模型评估，结果显示相比现有的 im2row/im2col 基于优化技术，可在全网络中提高 60% 左右的性能。