Early Exit Neural Networks (EENNs) endow astandard Deep Neural Network (DNN)
with Early Exit Classifiers (EECs), to provide predictions at intermediate
points of the processing when enough confidence in classification is achieved.
This leads to many benefits in terms of effectiveness and efficiency.
Currently, the design of EENNs is carried out manually by experts, a complex
and time-consuming task that requires accounting for many aspects, including
the correct placement, the thresholding, and the computational overhead of the
EECs. For this reason, the research is exploring the use of Neural Architecture
Search (NAS) to automatize the design of EENNs. Currently, few comprehensive
NAS solutions for EENNs have been proposed in the literature, and a fully
automated, joint design strategy taking into consideration both the backbone
and the EECs remains an open problem. To this end, this work presents Neural
Architecture Search for Hardware Constrained Early Exit Neural Networks
(NACHOS), the first NAS framework for the design of optimal EENNs satisfying
constraints on the accuracy and the number of Multiply and Accumulate (MAC)
operations performed by the EENNs at inference time. In particular, this
provides the joint design of backbone and EECs to select a set of admissible
(i.e., respecting the constraints) Pareto Optimal Solutions in terms of best
tradeoff between the accuracy and number of MACs. The results show that the
models designed by NACHOS are competitive with the state-of-the-art EENNs.
Additionally, this work investigates the effectiveness of two novel
regularization terms designed for the optimization of the auxiliary classifiers
of the EENN

通过神经架构搜索，设计硬件受限的早退出神经网络（NACHOS），在考虑准确度和乘加操作数量的约束下，实现了主干和辅助分类器的联合设计，以在准确度和乘加操作数量之间取得最佳权衡。NACHOS 设计的模型具有与最先进的早退神经网络竞争力的结果。

NACHOS: 硬件受限早期中止神经网络的神经架构搜索

NACHOS: Neural Architecture Search for Hardware Constrained Early Exit  Neural Networks

Deep neural networks (DNNs) have been successfully applied in various fields.
A major challenge of deploying DNNs, especially on edge devices, is power
consumption, due to the large number of multiply-and-accumulate (MAC)
operations. To address this challenge, we propose PowerPruning, a novel method
to reduce power consumption in digital neural network accelerators by selecting
weights that lead to less power consumption in MAC operations. In addition, the
timing characteristics of the selected weights together with all activation
transitions are evaluated. The weights and activations that lead to small
delays are further selected. Consequently, the maximum delay of the sensitized
circuit paths in the MAC units is reduced even without modifying MAC units,
which thus allows a flexible scaling of supply voltage to reduce power
consumption further. Together with retraining, the proposed method can reduce
power consumption of DNNs on hardware by up to 78.3% with only a slight
accuracy loss.

本文提出 PowerPruning 方法，通过选择在 MAC 操作中消耗更少功率的权重，以及考虑所选择权重的时序特征和所有激活转移，进一步选择导致小延迟的权重和激活，从而在不修改 MAC 单位的情况下减小了 MAC 单位敏化电路路径的最大延迟，实现了进一步的供应电压灵活缩放，使得该方法能够在仅有轻微精度损失的情况下，最多可将硬件上深度神经网络的功耗降低 78.3%。