We consider the quantization of deep neural networks (DNNs) to produce low-precision models for efficient inference of fixed-point operations. Compared to previous approaches to training quantized DNNs directly under the constraints of low-precision weights and activations, we learn the quantization of DNNs with minimal quantization loss through regularization. In particular, we introduce the learnable regularization coefficient to find accurate low-precision models efficiently in training. In our experiments, the proposed scheme yields the state-of-the-art low-precision models of AlexNet and ResNet-18, which have better accuracy than their previously available low-precision models. We also examine our quantization method to produce low-precision DNNs for image super resolution. We observe only $0.5$~dB peak signal-to-noise ratio (PSNR) loss when using binary weights and 8-bit activations. The proposed scheme can be used to train low-precision models from scratch or to fine-tune a well-trained high-precision model to converge to a low-precision model. Finally, we discuss how a similar regularization method can be adopted in DNN weight pruning and compression, and show that $401\times$ compression is achieved for LeNet-5.

本文提出了一种使用低精度权重和操作的DNN学习方法，利用可学习的正则化系数来加强高精度权重收敛到量化值的能力，并研究了如何通过权重剪枝、量化和熵编码来建立低精度DNN压缩管道，实验结果显示该方法可以在ImageNet分类和图像超分辨率网络的任务中取得与准确性相对可接受的最新压缩比。

可学习正则化的稀疏低精度神经网络学习