Data quantization is an effective method to accelerate neural network training and reduce power consumption. However, it is challenging to perform low-bit quantized training: the conventional equal-precision quantization will lead to either high accuracy loss or limited bit-width reduction, while existing mixed-precision methods offer high compression potential but failed to perform accurate and efficient bit-width assignment. In this work, we propose DYNASTY, a block-wise dynamic-precision neural network training framework. DYNASTY provides accurate data sensitivity information through fast online analytics, and maintains stable training convergence with an adaptive bit-width map generator. Network training experiments on CIFAR-100 and ImageNet dataset are carried out, and compared to 8-bit quantization baseline, DYNASTY brings up to $5.1\times$ speedup and $4.7\times$ energy consumption reduction with no accuracy drop and negligible hardware overhead.

提出了一种动态精度神经网络训练框架 DYNASTY，通过快速在线分析提供准确的数据敏感度信息并通过自适应位宽映射生成器维持稳定的训练收敛性，与传统 8 位量化相比，DYNASTY 在 CIFAR-100 和 ImageNet 数据集上带来高达 5.1 倍的加速和 4.7 倍的能耗降低，无误差降低及可忽略的硬件开销。

基于分块动态精度神经网络培训加速的在线量化敏感性分析