The performance of mobile AI accelerators has been evolving rapidly in the
past two years, nearly doubling with each new generation of SoCs. The current
4th generation of mobile NPUs is already approaching the results of
CUDA-compatible Nvidia graphics cards presented not long ago, which together
with the increased capabilities of mobile deep learning frameworks makes it
possible to run complex and deep AI models on mobile devices. In this paper, we
evaluate the performance and compare the results of all chipsets from Qualcomm,
HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware
acceleration for AI inference. We also discuss the recent changes in the
Android ML pipeline and provide an overview of the deployment of deep learning
models on mobile devices. All numerical results provided in this paper can be
found and are regularly updated on the official project website:
this http URL

本文评估了高通、海思、三星、联发科和紫光展锐提供的 AI 推理硬件加速的所有芯片组的性能，并比较了它们的结果。同时，我们还讨论了 Android ML 流水线的最近变化，并提供了在移动设备上部署深度学习模型的概述。