Deep convolutional neural networks (CNN) has become the most promising method
for object recognition, repeatedly demonstrating record breaking results for
image classification and object detection in recent years. However, a very deep
CNN generally involves many layers with millions of parameters, making the
storage of the network model to be extremely large. This prohibits the usage of
deep CNNs on resource limited hardware, especially cell phones or other
embedded devices. In this paper, we tackle this model storage issue by
investigating information theoretical vector quantization methods for
compressing the parameters of CNNs. In particular, we have found in terms of
compressing the most storage demanding dense connected layers, vector
quantization methods have a clear gain over existing matrix factorization
methods. Simply applying k-means clustering to the weights or conducting
product quantization can lead to a very good balance between model size and
recognition accuracy. For the 1000-category classification task in the ImageNet
challenge, we are able to achieve 16-24 times compression of the network with
only 1% loss of classification accuracy using the state-of-the-art CNN.

本文探讨信息理论中的向量量化方法，用于压缩卷积神经网络中的参数，结果表明针对最消耗存储的稠密连接层使用向量量化方法比现有的矩阵分解方法更加高效，仅使用最先进的 CNN 模型在 ImageNet 挑战赛的 1000 分类任务中，即使损失 1% 的分类准确率，也能获得 16-24 倍的网络压缩优化。