Optimizing resource utilization in target platforms is key to achieving high performance during DNN inference. While optimizations have been proposed for inference latency, memory footprint, and energy consumption, prior hardware-aware neural architecture search (NAS) methods have omitted resource utilization, preventing DNNs to take full advantage of the target inference platforms. Modeling resource utilization efficiently and accurately is challenging, especially for widely-used array-based inference accelerators such as Google TPU. In this work, we propose a novel hardware-aware NAS framework that does not only optimize for task accuracy and inference latency, but also for resource utilization. We also propose and validate a new computational model for resource utilization in inference accelerators. By using the proposed NAS framework and the proposed resource utilization model, we achieve 2.8 - 4x speedup for DNN inference compared to prior hardware-aware NAS methods while attaining similar or improved accuracy in image classification on CIFAR-10 and Imagenet-100 datasets.

本文提出了一种新的硬件感知神经结构搜索框架，旨在优化目标平台上的任务精度、推理延迟和资源利用。通过使用该框架和新的资源利用模型，我们在CIFAR-10和Imagenet-100数据集上实现了 2.8-4倍的DNN推理加速，并获得了与以往硬件感知神经结构搜索方法相似或更好的图像分类精度。

U-Boost NAS: 利用率提升的可微神经结构搜索