Depthwise separable convolution has shown great efficiency in network design,
but requires time-consuming training procedure with full training-set
available. This paper first analyzes the mathematical relationship between
regular convolutions and depthwise separable convolutions, and proves that the
former one could be approximated with the latter one in closed form. We show
depthwise separable convolutions are principal components of regular
convolutions. And then we propose network decoupling (ND), a training-free
method to accelerate convolutional neural networks (CNNs) by transferring
pre-trained CNN models into the MobileNet-like depthwise separable convolution
structure, with a promising speedup yet negligible accuracy loss. We further
verify through experiments that the proposed method is orthogonal to other
training-free methods like channel decomposition, spatial decomposition, etc.
Combining the proposed method with them will bring even larger CNN speedup. For
instance, ND itself achieves about 2X speedup for the widely used VGG16, and
combined with other methods, it reaches 3.7X speedup with graceful accuracy
degradation. We demonstrate that ND is widely applicable to classification
networks like ResNet, and object detection network like SSD300.

该论文提出了一种称为网络解耦（ND）的训练方法，可以将已有的卷积神经网络模型转换到 MobileNet 等深度可分离卷积结构中，进而加速 CNN，减少严重的准确度下降。通过实验，作者证明了 ND 与其他训练 - free 方法正交，可以更大程度的加速 CNN。