Hardware-friendly network quantization (e.g., binary/uniform quantization)
can efficiently accelerate the inference and meanwhile reduce memory
consumption of the deep neural networks, which is crucial for model deployment
on resource-limited devices like mobile phones. However, due to the
discreteness of low-bit quantization, existing quantization methods often face
the unstable training process and severe performance degradation. To address
this problem, in this paper we propose Differentiable Soft Quantization (DSQ)
to bridge the gap between the full-precision and low-bit networks. DSQ can
automatically evolve during training to gradually approximate the standard
quantization. Owing to its differentiable property, DSQ can help pursue the
accurate gradients in backward propagation, and reduce the quantization loss in
forward process with an appropriate clipping range. Extensive experiments over
several popular network structures show that training low-bit neural networks
with DSQ can consistently outperform state-of-the-art quantization methods.
Besides, our first efficient implementation for deploying 2 to 4-bit DSQ on
devices with ARM architecture achieves up to 1.7$\times$ speed up, compared
with the open-source 8-bit high-performance inference framework NCNN. [31]

本研究提出了 Differentiable Soft Quantization（DSQ）来解决低位量化的不稳定训练和性能降低问题，通过在训练过程中逐渐逼近标准量化值，可以帮助追求精确的反向传播和减少正向传递中的量化损失，同时能在 ARM 架构的设备上实现 2 到 4 位 DSQ，并且与现有高性能推理框架相比，速度提高了 1.7 倍。

可微软量化：架起全精度和低比特神经网络的桥梁

Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit  Neural Networks

Recent work has shown that fast, compact low-bitwidth neural networks can be
surprisingly accurate. These networks use homogeneous binarization: all
parameters in each layer or (more commonly) the whole model have the same low
bitwidth (e.g., 2 bits). However, modern hardware allows efficient designs
where each arithmetic instruction can have a custom bitwidth, motivating
heterogeneous binarization, where every parameter in the network may have a
different bitwidth. In this paper, we show that it is feasible and useful to
select bitwidths at the parameter granularity during training. For instance a
heterogeneously quantized version of modern networks such as AlexNet and
MobileNet, with the right mix of 1-, 2- and 3-bit parameters that average to
just 1.4 bits can equal the accuracy of homogeneous 2-bit versions of these
networks. Further, we provide analyses to show that the heterogeneously
binarized systems yield FPGA- and ASIC-based implementations that are
correspondingly more efficient in both circuit area and energy efficiency than
their homogeneous counterparts.

本文讨论了低比特神经网络在同质二值化和异质二值化两种情况下的表现，提出使用参数粒度量化的异质二值化方法，其在减小模型位宽并保证性能稳定方面具有优势。