Mobile and embedded platforms are increasingly required to efficiently
execute computationally demanding DNNs across heterogeneous processing
elements. At runtime, the available hardware resources to DNNs can vary
considerably due to other concurrently running applications. The performance
requirements of the applications could also change under different scenarios.
To achieve the desired performance, dynamic DNNs have been proposed in which
the number of channels/layers can be scaled in real time to meet different
requirements under varying resource constraints. However, the training process
of such dynamic DNNs can be costly, since platform-aware models of different
deployment scenarios must be retrained to become dynamic. This paper proposes
Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware
NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family
of sub-networks from a static OFA backbone model, and contains a runtime
manager to choose different sub-networks under different runtime environments.
As such, Dynamic-OFA does not need the traditional dynamic DNN training
pipeline. Compared to the state-of-the-art, our experimental results using
ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x
(GPU) faster for similar ImageNet Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU)
higher accuracy at similar latency.

本研究论文探讨了动态深度神经网络在不同运行环境下实现快速性能的问题，并提出了一种新的动态 DNN 方法 Dynamic-OFA，该方法通过预先采样 OFA 模型中的子网，配合运行时管理器在不同的运行环境中选择不同的子网来实现。在实验中，使用 ImageNet 数据集验证了该方法的性能表现，相比已有方法，Dynamic-OFA 在 Jetson Xavier NX 上可以实现最高 3.5x 的 CPU 速度和 2.4x 的 GPU 速度提升，并相应地实现 3.8％的 CPU 精度和 5.1％的 GPU 精度提升。