Fixed-point optimization of deep neural networks plays an important role in
hardware based design and low-power implementations. Many deep neural networks
show fairly good performance even with 2- or 3-bit precision when quantized
weights are fine-tuned by retraining. We propose an improved fixedpoint
optimization algorithm that estimates the quantization step size dynamically
during the retraining. In addition, a gradual quantization scheme is also
tested, which sequentially applies fixed-point optimizations from high- to
low-precision. The experiments are conducted for feed-forward deep neural
networks (FFDNNs), convolutional neural networks (CNNs), and recurrent neural
networks (RNNs).

本文提出了一种对定点优化算法的改进，它可以动态估计量化步长，同时实现了逐步量化方案，经实验发现：该算法适用于前馈神经网络、卷积神经网络和循环神经网络。

使用自适应步长重新训练的深度神经网络的固定点优化

Fixed-point optimization of deep neural networks with adaptive step size  retraining

Recurrent neural networks have shown excellent performance in many
applications, however they require increased complexity in hardware or software
based implementations. The hardware complexity can be much lowered by
minimizing the word-length of weights and signals. This work analyzes the
fixed-point performance of recurrent neural networks using a retrain based
quantization method. The quantization sensitivity of each layer in RNNs is
studied, and the overall fixed-point optimization results minimizing the
capacity of weights while not sacrificing the performance are presented. A
language model and a phoneme recognition examples are used.

本研究通过基于再训练的量化方法分析了循环神经网络的定点性能，研究了 RNN 每一层的量化敏感度，并提出了能够在不损失性能的情况下最小化权重容量的整体定点优化结果。实验使用了语言模型和音素识别示例。