Mobile vision systems such as smartphones, drones, and augmented-reality
headsets are revolutionizing our lives. These systems usually run multiple
applications concurrently and their available resources at runtime are dynamic
due to events such as starting new applications, closing existing applications,
and application priority changes. In this paper, we present NestDNN, a
framework that takes the dynamics of runtime resources into account to enable
resource-aware multi-tenant on-device deep learning for mobile vision systems.
NestDNN enables each deep learning model to offer flexible resource-accuracy
trade-offs. At runtime, it dynamically selects the optimal resource-accuracy
trade-off for each deep learning model to fit the model's resource demand to
the system's available runtime resources. In doing so, NestDNN efficiently
utilizes the limited resources in mobile vision systems to jointly maximize the
performance of all the concurrently running applications. Our experiments show
that compared to the resource-agnostic status quo approach, NestDNN achieves as
much as 4.2% increase in inference accuracy, 2.0x increase in video frame
processing rate and 1.7x reduction on energy consumption.

本文介绍了一个名为 NestDNN 的框架，它考虑到了运行时资源的动态性，能够为移动视觉系统提供只能多租户的深度学习。NestDNN 使每个深度学习模型提供灵活的资源 - 准确性权衡，并在运行时为每个深度学习模型动态选择最佳的资源 - 准确性权衡，以适应模型的资源需求。与资源不可知的现状方法相比，实验表明，NestDNN 在推断准确性上提高了 4.2％，视频帧处理速度提高了 2.0 倍，能源消耗降低了 1.7 倍。