Despite the success of deep neural networks (DNNs), state-of-the-art models
are too large to deploy on low-resource devices or common server configurations
in which multiple models are held in memory. Model compression methods address
this limitation by reducing the memory footprint, latency, or energy
consumption of a model with minimal impact on accuracy. We focus on the task of
reducing the number of learnable variables in the model. In this work we
combine ideas from weight hashing and dimensionality reductions resulting in a
simple and powerful structured multi-hashing method based on matrix products
that allows direct control of model size of any deep network and is trained
end-to-end. We demonstrate the strength of our approach by compressing models
from the ResNet, EfficientNet, and MobileNet architecture families. Our method
allows us to drastically decrease the number of variables while maintaining
high accuracy. For instance, by applying our approach to EfficentNet-B4 (16M
parameters) we reduce it to to the size of B0 (5M parameters), while gaining
over 3% in accuracy over B0 baseline. On the commonly used benchmark CIFAR10 we
reduce the ResNet32 model by 75% with no loss in quality, and are able to do a
10x compression while still achieving above 90% accuracy.

本文介绍了一种基于哈希和降维的结构化多哈希方法，可用于控制任何深度网络的模型大小，从而大幅减少可学习变量且依旧保持较高的精度。作者在 ResNet，EfficientNet 和 MobileNet 等结构中进行了压缩，并且在 CIFAR10 数据集上仅损失 10％而使 ResNet32 模型减少了 75％。