We propose a simple two-step approach for speeding up convolution layers
within large convolutional neural networks based on tensor decomposition and
discriminative fine-tuning. Given a layer, we use non-linear least squares to
compute a low-rank CP-decomposition of the 4D convolution kernel tensor into a
sum of a small number of rank-one tensors. At the second step, this
decomposition is used to replace the original convolutional layer with a
sequence of four convolutional layers with small kernels. After such
replacement, the entire network is fine-tuned on the training data using
standard backpropagation process.
We evaluate this approach on two CNNs and show that it is competitive with
previous approaches, leading to higher obtained CPU speedups at the cost of
lower accuracy drops for the smaller of the two networks. Thus, for the
36-class character classification CNN, our approach obtains a 8.5x CPU speedup
of the whole network with only minor accuracy drop (1% from 91% to 90%). For
the standard ImageNet architecture (AlexNet), the approach speeds up the second
convolution layer by a factor of 4x at the cost of $1\%$ increase of the
overall top-5 classification error.

该研究提出一种使用张量分解和有区别的微调方法来加速卷积神经网络中的卷积层，并且在两个 CNN 上进行验证并表明它具有与以前方法相竞争的性能，能够获得较高的 CPU 加速和较小的精度下降，这对于图像分类具有很大帮助。