Convolutional neural networks (CNNs) have been increasingly deployed to edge
devices. Hence, many efforts have been made towards efficient CNN inference in
resource-constrained platforms. This paper attempts to explore an orthogonal
direction: how to conduct more energy-efficient training of CNNs, so as to
enable on-device training. We strive to reduce the energy cost during training,
by dropping unnecessary computations from three complementary levels:
stochastic mini-batch dropping on the data level; selective layer update on the
model level; and sign prediction for low-cost, low-precision back-propagation,
on the algorithm level. Extensive simulations and ablation studies, with real
energy measurements from an FPGA board, confirm the superiority of our proposed
strategies and demonstrate remarkable energy savings for training. For example,
when training ResNet-74 on CIFAR-10, we achieve aggressive energy savings of
>90% and >60%, while incurring a top-1 accuracy loss of only about 2% and 1.2%,
respectively. When training ResNet-110 on CIFAR-100, an over 84% training
energy saving is achieved without degrading inference accuracy.

通过三种方法 (stochastic mini-batch dropping, selective layer update, sign prediction) 来减少训练过程中的计算并实现在资源受限的设备上进行训练，该方法能够在保持准确率损失相对较小 (不超过 2%) 的前提下，实现能耗降低超过 90%。