The advent of tiny AI accelerators opens opportunities for deep neural
network deployment at the extreme edge, offering reduced latency, lower power
cost, and improved privacy in on-device ML inference. Despite these
advancements, challenges persist due to inherent limitations of these
accelerators, such as restricted onboard memory and single-device focus. This
paper introduces Synergy, a system that dynamically composes tiny AI
accelerators for multi-tenant models, effectively addressing tinyML's critical
challenges for the increasing demand for on-device AI. A key feature of Synergy
is its virtual computing space, providing a unified, virtualized view of
resources and enabling efficient task mapping to physical devices. Synergy's
runtime orchestration module ensures optimal inference across dynamic and
heterogeneous accelerators. Our evaluations with 7 baselines and 8 models
demonstrate that Synergy improves throughput by an average of 8.0X compared to
baselines.

通过动态组合微型 AI 加速器，Synergy 系统有效解决了 on-device AI 日益增长的需求对 tinyML 的关键挑战，其虚拟计算空间和运行时协调模块能够提供统一的资源视图和优化的推理，可将吞吐量提高了 8.0 倍。