Deep neural networks (DNNs) have been successfully applied in various fields.
In DNNs, a large number of multiply-accumulate (MAC) operations is required to
be performed, posing critical challenges in applying them in
resource-constrained platforms, e.g., edge devices. Dynamic neural networks
have been introduced to allow a structural adaption, e.g., early-exit,
according to different inputs to reduce the computational cost of DNNs.
Existing early-exit techniques deploy classifiers at intermediate layers of
DNNs to push them to make a classification decision as early as possible.
However, the learned features at early layers might not be sufficient to
exclude all the irrelevant classes and decide the correct class, leading to
suboptimal results. To address this challenge, in this paper, we propose a
class-based early-exit for dynamic inference. Instead of pushing DNNs to make a
dynamic decision at intermediate layers, we take advantages of the learned
features in these layers to exclude as many irrelevant classes as possible, so
that later layers only have to determine the target class among the remaining
classes. Until at a layer only one class remains, this class is the
corresponding classification result. To realize this class-based exclusion, we
assign each class with a classifier at intermediate layers and train the
networks together with these classifiers. Afterwards, an exclusion strategy is
developed to exclude irrelevant classes at early layers. Experimental results
demonstrate the computational cost of DNNs in inference can be reduced
significantly.

该研究论文介绍了一种基于类别排除的动态推理方法，通过在中间层分配分类器并利用其学习到的特征排除尽可能多的无关类别，从而显著降低深度神经网络在推理中的计算成本。