Extreme edge platforms, such as in-vehicle smart devices, require efficient
deployment of quantized deep neural networks (DNNs) to enable intelligent
applications with limited amounts of energy, memory, and computing resources.
However, many edge devices struggle to boost inference throughput of various
quantized DNNs due to the varying quantization levels, and these devices lack
floating-point (FP) support for on-device learning, which prevents them from
improving model accuracy while ensuring data privacy. To tackle the challenges
above, we propose a precision-scalable RISC-V DNN processor with on-device
learning capability. It facilitates diverse precision levels of fixed-point DNN
inference, spanning from 2-bit to 16-bit, and enhances on-device learning
through improved support with FP16 operations. Moreover, we employ multiple
methods such as FP16 multiplier reuse and multi-precision integer multiplier
reuse, along with balanced mapping of FPGA resources, to significantly improve
hardware resource utilization. Experimental results on the Xilinx ZCU102 FPGA
show that our processor significantly improves inference throughput by
1.6$\sim$14.6$\times$ and energy efficiency by 1.1$\sim$14.6$\times$ across
various DNNs, compared to the prior art, XpulpNN. Additionally, our processor
achieves a 16.5$\times$ higher FP throughput for on-device learning.

提出了一种支持多种精度的定点深度神经网络推断和增强设备上学习能力的精度可伸缩的 RISC-V DNN 处理器，通过改进硬件资源利用率，显著提高推断吞吐量和能效，并实现 16.5 倍更高的设备上学习的浮点吞吐量。

一种具有极限边缘智能设备学习能力的精确可扩展 RISC-V DNN 处理器

A Precision-Scalable RISC-V DNN Processor with On-Device Learning  Capability at the Extreme Edge

Quantized deep neural networks (QDNNs) are attractive due to their much lower
memory storage and faster inference speed than their regular full precision
counterparts. To maintain the same performance level especially at low
bit-widths, QDNNs must be retrained. Their training involves piecewise constant
activation functions and discrete weights, hence mathematical challenges arise.
We introduce the notion of coarse gradient and propose the blended coarse
gradient descent (BCGD) algorithm, for training fully quantized neural
networks. Coarse gradient is generally not a gradient of any function but an
artificial ascent direction. The weight update of BCGD goes by coarse gradient
correction of a weighted average of the full precision weights and their
quantization (the so-called blending), which yields sufficient descent in the
objective value and thus accelerates the training. Our experiments demonstrate
that this simple blending technique is very effective for quantization at
extremely low bit-width such as binarization. In full quantization of ResNet-18
for ImageNet classification task, BCGD gives 64.36\% top-1 accuracy with binary
weights across all layers and 4-bit adaptive activation. If the weights in the
first and last layers are kept in full precision, this number increases to
65.46\%. As theoretical justification, we show convergence analysis of coarse
gradient descent for a two-linear-layer neural network model with Gaussian
input data, and prove that the expected coarse gradient correlates positively
with the underlying true gradient.

提出了混合粗微分下降算法 (BCGD)，该算法利用了人工升高方向的粗微分来进行系数的权重更新，从而在目标值上产生足够的下降，加速了量化训练效果，并在 ImageNet 分类任务中全量化 ResNet-18 网络汇报了 64.36％的 top-1 准确率。