Researchers have long touted a vision of the future enabled by a
proliferation of internet-of-things devices, including smart sensors, homes,
and cities. Increasingly, embedding intelligence in such devices involves the
use of deep neural networks. However, their storage and processing requirements
make them prohibitive for cheap, off-the-shelf platforms. Overcoming those
requirements is necessary for enabling widely-applicable smart devices. While
many ways of making models smaller and more efficient have been developed,
there is a lack of understanding of which ones are best suited for particular
scenarios. More importantly for edge platforms, those choices cannot be
analyzed in isolation from cost and user experience. In this work, we
holistically explore how quantization, model scaling, and multi-modality
interact with system components such as memory, sensors, and processors. We
perform this hardware/software co-design from the cost, latency, and
user-experience perspective, and develop a set of guidelines for optimal system
design and model deployment for the most cost-constrained platforms. We
demonstrate our approach using an end-to-end, on-device, biometric user
authentication system using a $20 ESP-EYE board.

通过量化、成本限制平台等因素综合考虑，探讨如何在边缘平台上最优化地设计系统和部署模型，以实现广泛适用的智能设备。

基于成本的机器学习流水线硬件软件协同优化

Cost-Driven Hardware-Software Co-Optimization of Machine Learning  Pipelines

Developing deep learning models for resource-constrained Internet-of-Things
(IoT) devices is challenging, as it is difficult to achieve both good quality
of results (QoR), such as DNN model inference accuracy, and quality of service
(QoS), such as inference latency, throughput, and power consumption. Existing
approaches typically separate the DNN model development step from its
deployment on IoT devices, resulting in suboptimal solutions. In this paper, we
first introduce a few interesting but counterintuitive observations about such
a separate design approach, and empirically show why it may lead to suboptimal
designs. Motivated by these observations, we then propose a novel and practical
bi-directional co-design approach: a bottom-up DNN model design strategy
together with a top-down flow for DNN accelerator design. It enables a joint
optimization of both DNN models and their deployment configurations on IoT
devices as represented as FPGAs. We demonstrate the effectiveness of the
proposed co-design approach on a real-life object detection application using
Pynq-Z1 embedded FPGA. Our method obtains the state-of-the-art results on both
QoR with high accuracy (IoU) and QoS with high throughput (FPS) and high energy
efficiency.

本文提出一种底层深度神经网络模型设计策略和顶层 DNN 加速器设计流程的双向协同设计方法，通过在 IoT 设备上进行联合优化使得既能获得高 IoU 精度的 QoR 也能获得高 FPS 和高能效的 QoS。