Batteryless systems frequently face power failures, requiring extra runtime
buffers to maintain inference progress and leaving only a memory space for
storing ultra-tiny deep neural networks (DNNs). Besides, making these models
responsive to stochastic energy harvesting dynamics during inference requires a
balance between inference accuracy, latency, and energy overhead. Recent works
on compression mostly focus on time and memory, but often ignore energy
dynamics or significantly reduce the accuracy of pre-trained DNNs. Existing
energy-adaptive inference works modify the architecture of pre-trained models
and have significant memory overhead. Thus, energy-adaptive and accurate
inference of pre-trained DNNs on batteryless devices with extreme memory
constraints is more challenging than traditional microcontrollers. We combat
these issues by proposing FreeML, a framework to optimize pre-trained DNN
models for memory-efficient and energy-adaptive inference on batteryless
systems. FreeML comprises (1) a novel compression technique to reduce the model
footprint and runtime memory requirements simultaneously, making them
executable on extremely memory-constrained batteryless platforms; and (2) the
first early exit mechanism that uses a single exit branch for all exit points
to terminate inference at any time, making models energy-adaptive with minimal
memory overhead. Our experiments showed that FreeML reduces the model sizes by
up to $95 \times$, supports adaptive inference with a $2.03-19.65 \times$ less
memory overhead, and provides significant time and energy benefits with only a
negligible accuracy drop compared to the state-of-the-art.

提出了一种名为 FreeML 的框架，旨在优化预训练的深度神经网络模型，以在电池无需设备上进行内存高效和能量自适应的推理，通过使用压缩技术减小模型尺寸，并引入早期终止机制以实现能量自适应推理，同时最小化内存开销。