Executing machine learning workloads locally on resource constrained
microcontrollers (MCUs) promises to drastically expand the application space of
IoT. However, so-called TinyML presents severe technical challenges, as deep
neural network inference demands a large compute and memory budget. To address
this challenge, neural architecture search (NAS) promises to help design
accurate ML models that meet the tight MCU memory, latency and energy
constraints. A key component of NAS algorithms is their latency/energy model,
i.e., the mapping from a given neural network architecture to its inference
latency/energy on an MCU. In this paper, we observe an intriguing property of
NAS search spaces for MCU model design: on average, model latency varies
linearly with model operation (op) count under a uniform prior over models in
the search space. Exploiting this insight, we employ differentiable NAS (DNAS)
to search for models with low memory usage and low op count, where op count is
treated as a viable proxy to latency. Experimental results validate our
methodology, yielding our MicroNet models, which we deploy on MCUs using
Tensorflow Lite Micro, a standard open-source NN inference runtime widely used
in the TinyML community. MicroNets demonstrate state-of-the-art results for all
three TinyMLperf industry-standard benchmark tasks: visual wake words, audio
keyword spotting, and anomaly detection. Models and training scripts can be
found at github.com/ARM-software/ML-zoo.

本文介绍了机器学习在资源受限的微控制器上的应用，使用神经结构搜索算法来设计满足内存、延迟和能耗限制的模型，并通过可微分神经结构搜索算法搜索出了具有低内存使用和低操作计数的 MicroNet 模型，验证该文述方法在视觉唤醒词、音频关键词检测和异常检测等领域的优越性。