The increasing interest in TinyML, i.e., near-sensor machine learning on
power budgets of a few tens of mW, is currently pushing toward enabling
TinyML-class training as opposed to inference only. Current training
algorithms, based on various forms of error and gradient backpropagation, rely
on floating-point matrix operations to meet the precision and dynamic range
requirements. So far, the energy and power cost of these operations has been
considered too high for TinyML scenarios. This paper addresses the open
challenge of near-sensor training on a few mW power budget and presents RedMulE
- Reduced-Precision Matrix Multiplication Engine, a low-power specialized
accelerator conceived for multi-precision floating-point General Matrix-Matrix
Operations (GEMM-Ops) acceleration, supporting FP16, as well as hybrid FP8
formats, with {sign, exponent, mantissa}=({1,4,3}, {1,5,2}). We integrate
RedMule into a Parallel Ultra-Low-Power (PULP) cluster containing eight
energy-efficient RISC-V cores sharing a tightly-coupled data memory and
implement the resulting system in a 22 nm technology. At its best efficiency
point (@ 470 MHz, 0.65 V), the RedMulE-augmented PULP cluster achieves 755
GFLOPS/W and 920 GFLOPS/W during regular General Matrix-Matrix Multiplication
(GEMM), and up to 1.19 TFLOPS/W and 1.67 TFLOPS/W when executing GEMM-Ops,
respectively, for FP16 and FP8 input/output tensors. In its best performance
point (@ 613 MHz, 0.8 V), RedMulE achieves up to 58.5 GFLOPS and 117 GFLOPS for
FP16 and FP8, respectively, with 99.4% utilization of the array of Computing
Elements and consuming less than 60 mW on average, thus enabling on-device
training of deep learning models in TinyML application scenarios while
retaining the flexibility to tackle other classes of common linear algebra
problems efficiently.

该研究旨在解决 TinyML 场景下近传感器低功耗训练的挑战，并提出了一种低功耗特殊加速器 RedMulE，它支持 FP16 和混合 FP8 格式的多精度浮点通用矩阵乘法运算（GEMM-Ops）加速，并成功地使小型机器学习模型在低功耗设备上训练。