The high computation, memory, and power budgets of inferring convolutional
neural networks (CNNs) are major bottlenecks of model deployment to edge
computing platforms, e.g., mobile devices and IoT. Moreover, training CNNs is
time and energy-intensive even on high-grade servers. Convolution layers and
fully connected layers, because of their intense use of multiplications, are
the dominant contributor to this computation budget.
We propose to alleviate this problem by introducing two new operations:
convolutional shifts and fully-connected shifts which replace multiplications
with bitwise shift and sign flipping during both training and inference. During
inference, both approaches require only 5 bits (or less) to represent the
weights. This family of neural network architectures (that use convolutional
shifts and fully connected shifts) is referred to as DeepShift models. We
propose two methods to train DeepShift models: DeepShift-Q which trains regular
weights constrained to powers of 2, and DeepShift-PS that trains the values of
the shifts and sign flips directly.
Very close accuracy, and in some cases higher accuracy, to baselines are
achieved. Converting pre-trained 32-bit floating-point baseline models of
ResNet18, ResNet50, VGG16, and GoogleNet to DeepShift and training them for 15
to 30 epochs, resulted in Top-1/Top-5 accuracies higher than that of the
original model.
Last but not least, we implemented the convolutional shifts and fully
connected shift GPU kernels and showed a reduction in latency time of 25% when
inferring ResNet18 compared to unoptimized multiplication-based GPU kernels.
The code can be found at this https URL

提出使用卷积位移和完全连接位移加速模型部署到边缘计算平台。在使用 DeepShift 模型转换和训练的预先训练的 ResNet18，ResNet50，VGG16 和 GoogleNet 模型中，可以获得接近甚至更高的准确率，并且 GPU 内核可以减少 25% 的延迟时间，具有较好的应用前景。