It has been proven that, compared to using 32-bit floating-point numbers in
the training phase, Deep Convolutional Neural Networks (DCNNs) can operate with
low precision during inference, thereby saving memory space and power
consumption. However, quantizing networks is always accompanied by an accuracy
decrease. Here, we propose a method, double-stage Squeeze-and-Threshold
(double-stage ST). It uses the attention mechanism to quantize networks and
achieve state-of-art results. Using our method, the 3-bit model can achieve
accuracy that exceeds the accuracy of the full-precision baseline model. The
proposed double-stage ST activation quantization is easy to apply: inserting it
before the convolution.

本文提出双阶段压缩与阈值化方法 (double-stage ST)，使用注意力机制量化网络，在卷积前插入即可实现，达到超越全精度基线模型的准确性。

卷积神经网络量化与注意力机制

Convolutional Neural Networks Quantization with Attention

This paper proposes three simple, compact yet effective representations of
depth sequences, referred to respectively as Dynamic Depth Images (DDI),
Dynamic Depth Normal Images (DDNI) and Dynamic Depth Motion Normal Images
(DDMNI), for both isolated and continuous action recognition. These dynamic
images are constructed from a segmented sequence of depth maps using
hierarchical bidirectional rank pooling to effectively capture the
spatial-temporal information. Specifically, DDI exploits the dynamics of
postures over time and DDNI and DDMNI exploit the 3D structural information
captured by depth maps. Upon the proposed representations, a ConvNet based
method is developed for action recognition. The image-based representations
enable us to fine-tune the existing Convolutional Neural Network (ConvNet)
models trained on image data without training a large number of parameters from
scratch. The proposed method achieved the state-of-art results on three large
datasets, namely, the Large-scale Continuous Gesture Recognition Dataset (means
Jaccard index 0.4109), the Large-scale Isolated Gesture Recognition Dataset
(59.21%), and the NTU RGB+D Dataset (87.08% cross-subject and 84.22%
cross-view) even though only the depth modality was used.

本论文提出了三种简单、紧凑而有效的深度序列表示，称为 DDI、DDNI 和 DDMNI，用于孤立和连续行为识别。在这些动态图像上，设计了基于 ConvNet 的方法进行行动识别，取得了三个大型数据集上的最先进结果，即 Large-scale Continuous Gesture Recognition Dataset（Jaccard 指数 0.4109）、Large-scale Isolated Gesture Recognition Dataset（59.21％）和 NTU RGB + D Dataset（跨主体 87.08％和跨视角 84.22％），即使只使用深度模态。