Deploying neural networks on microcontroller units (MCUs) presents
substantial challenges due to their constrained computation and memory
resources. Previous researches have explored patch-based inference as a
strategy to conserve memory without sacrificing model accuracy. However, this
technique suffers from severe redundant computation overhead, leading to a
substantial increase in execution latency. A feasible solution to address this
issue is mixed-precision quantization, but it faces the challenges of accuracy
degradation and a time-consuming search time. In this paper, we propose
QuantMCU, a novel patch-based inference method that utilizes value-driven
mixed-precision quantization to reduce redundant computation. We first utilize
value-driven patch classification (VDPC) to maintain the model accuracy. VDPC
classifies patches into two classes based on whether they contain outlier
values. For patches containing outlier values, we apply 8-bit quantization to
the feature maps on the dataflow branches that follow. In addition, for patches
without outlier values, we utilize value-driven quantization search (VDQS) on
the feature maps of their following dataflow branches to reduce search time.
Specifically, VDQS introduces a novel quantization search metric that takes
into account both computation and accuracy, and it employs entropy as an
accuracy representation to avoid additional training. VDQS also adopts an
iterative approach to determine the bitwidth of each feature map to further
accelerate the search process. Experimental results on real-world MCU devices
show that QuantMCU can reduce computation by 2.2x on average while maintaining
comparable model accuracy compared to the state-of-the-art patch-based
inference methods.

提出了一种名为 QuantMCU 的新的基于补丁的推断方法，利用价值驱动的混合精度量化来减少冗余计算。通过利用 Value-Driven Patch Classification（VDPC）和 Value-Driven Quantization Search（VDQS）来维持模型的准确性并降低搜索时间。实验结果表明，QuantMCU 减少了 2.2 倍的计算量，同时与最先进的基于补丁的推断方法相比保持了可比的模型准确性。