Machine learning on tiny IoT devices based on microcontroller units (MCU) is
appealing but challenging: the memory of microcontrollers is 2-3 orders of
magnitude smaller even than mobile phones. We propose MCUNet, a framework that
jointly designs the efficient neural architecture (TinyNAS) and the lightweight
inference engine (TinyEngine), enabling ImageNet-scale inference on
microcontrollers. TinyNAS adopts a two-stage neural architecture search
approach that first optimizes the search space to fit the resource constraints,
then specializes the network architecture in the optimized search space.
TinyNAS can automatically handle diverse constraints (i.e.device, latency,
energy, memory) under low search costs.TinyNAS is co-designed with TinyEngine,
a memory-efficient inference library to expand the search space and fit a
larger model. TinyEngine adapts the memory scheduling according to the overall
network topology rather than layer-wise optimization, reducing the memory usage
by 4.8x, and accelerating the inference by 1.7-3.3x compared to TF-Lite Micro
and CMSIS-NN. MCUNet is the first to achieves >70% ImageNet top1 accuracy on an
off-the-shelf commercial microcontroller, using 3.5x less SRAM and 5.7x less
Flash compared to quantized MobileNetV2 and ResNet-18. On visual&audio wake
words tasks, MCUNet achieves state-of-the-art accuracy and runs 2.4-3.4x faster
than MobileNetV2 and ProxylessNAS-based solutions with 3.7-4.1x smaller peak
SRAM. Our study suggests that the era of always-on tiny machine learning on IoT
devices has arrived. Code and models can be found here: this https URL

本研究提出了 MCUNet 框架，该框架共同设计了高效的神经结构（TinyNAS）和轻量级推断引擎（TinyEngine），使得可以在微控制器上进行 ImageNet 规模的推理。