A low precision deep neural network training technique for producing sparse, ternary neural networks is presented. The technique incorporates hard- ware implementation costs during training to achieve significant model compression for inference. Training involves three stages: network training using L2 regularization and a quantization threshold regularizer, quantization pruning, and finally retraining. Resulting networks achieve improved accuracy, reduced memory footprint and reduced computational complexity compared with conventional methods, on MNIST and CIFAR10 datasets. Our networks are up to 98% sparse and 5 & 11 times smaller than equivalent binary and ternary models, translating to significant resource and speed benefits for hardware implementations.

本文提出了一种低精度深度神经网络训练技术，用于生成稀疏三元神经网络，该技术在训练期间考虑硬件实现成本，以实现重要的模型压缩，结果网络精度提高、内存占用和计算复杂性减少，与传统方法相比，在MNIST和CIFAR10数据集上，我们的网络稀疏度高达98％，比等价二进制和三进制模型小5＆11倍，从而在硬件实现方面具有重大资源和速度优势。

使用三值网络中的稀疏诱导正则化压缩低精度深度神经网络