Low-precision networks, with weights and activations quantized to low
bit-width, are widely used to accelerate inference on edge devices. However,
current solutions are uniform, using identical bit-width for all filters. This
fails to account for the different sensitivities of different filters and is
suboptimal. Mixed-precision networks address this problem, by tuning the
bit-width to individual filter requirements. In this work, the problem of
optimal mixed-precision network search (MPS) is considered. To circumvent its
difficulties of discrete search space and combinatorial optimization, a new
differentiable search architecture is proposed, with several novel
contributions to advance the efficiency by leveraging the unique properties of
the MPS problem. The resulting Efficient differentiable MIxed-Precision network
Search (EdMIPS) method is effective at finding the optimal bit allocation for
multiple popular networks, and can search a large model, e.g. Inception-V3,
directly on ImageNet without proxy task in a reasonable amount of time. The
learned mixed-precision networks significantly outperform their uniform
counterparts.

该研究旨在解决低精度网络中过滤器位宽不同敏感性也不同的问题，提出了一种新型可微分搜索架构 ——Efficient differentiable Mixed-Precision network Search（EdMIPS），采用该方法能够高效地寻找多种流行网络最佳位分配，且搜索速度快，学习到的混合精度网络性能显著优于其均匀位宽网络。

重新思考混合精度神经网络的可微分搜索

Rethinking Differentiable Search for Mixed-Precision Neural Networks

In this paper, we seek to tackle a challenge in training low-precision
networks: the notorious difficulty in propagating gradient through a
low-precision network due to the non-differentiable quantization function. We
propose a solution by training the low-precision network with a fullprecision
auxiliary module. Specifically, during training, we construct a mix-precision
network by augmenting the original low-precision network with the full
precision auxiliary module. Then the augmented mix-precision network and the
low-precision network are jointly optimized. This strategy creates additional
full-precision routes to update the parameters of the low-precision model, thus
making the gradient back-propagates more easily. At the inference time, we
discard the auxiliary module without introducing any computational complexity
to the low-precision network. We evaluate the proposed method on image
classification and object detection over various quantization approaches and
show consistent performance increase. In particular, we achieve near lossless
performance to the full-precision model by using a 4-bit detector, which is of
great practical value.

本文提出一种解决训练低精度神经网络中梯度传播困难的方法，即通过在训练期间将低精度网络与全精度辅助模块相结合进行混合精度训练，进而提高模型的性能。

使用全精度辅助模块训练量化神经网络

Training Quantized Neural Networks with a Full-precision Auxiliary  Module

To realize the promise of ubiquitous embedded deep network inference, it is
essential to seek limits of energy and area efficiency. To this end,
low-precision networks offer tremendous promise because both energy and area
scale down quadratically with the reduction in precision. Here we demonstrate
ResNet-18, -34, -50, -152, Inception-v3, Densenet-161, and VGG-16bn networks on
the ImageNet classification benchmark that, at 8-bit precision exceed the
accuracy of the full-precision baseline networks after one epoch of finetuning,
thereby leveraging the availability of pretrained models. We also demonstrate
ResNet-18, -34, -50, -152, Densenet-161, and VGG-16bn 4-bit models that match
the accuracy of the full-precision baseline networks -- the highest scores to
date. Surprisingly, the weights of the low-precision networks are very close
(in cosine similarity) to the weights of the corresponding baseline networks,
making training from scratch unnecessary.
We find that gradient noise due to quantization during training increases
with reduced precision, and seek ways to overcome this noise. The number of
iterations required by SGD to achieve a given training error is related to the
square of (a) the distance of the initial solution from the final plus (b) the
maximum variance of the gradient estimates. Therefore, we (a) reduce solution
distance by starting with pretrained fp32 precision baseline networks and
fine-tuning, and (b) combat gradient noise introduced by quantization by
training longer and reducing learning rates. Sensitivity analysis indicates
that these simple techniques, coupled with proper activation function range
calibration to take full advantage of the limited precision, are sufficient to
discover low-precision networks, if they exist, close to fp32 precision
baseline networks. The results herein provide evidence that 4-bits suffice for
classification.

通过调整激活函数范围并使用精细调整的预训练模型来组合简单技术，可以发现与 fp32 模型接近的低精度模型，同时提高了效率，本文证明了 4 位精度足以进行分类。

为高效嵌入式推断发现接近全精度网络的低精度网络

Discovering Low-Precision Networks Close to Full-Precision Networks for  Efficient Embedded Inference

Deep learning networks have achieved state-of-the-art accuracies on computer
vision workloads like image classification and object detection. The performant
systems, however, typically involve big models with numerous parameters. Once
trained, a challenging aspect for such top performing models is deployment on
resource constrained inference systems - the models (often deep networks or
wide networks or both) are compute and memory intensive. Low-precision numerics
and model compression using knowledge distillation are popular techniques to
lower both the compute requirements and memory footprint of these deployed
models. In this paper, we study the combination of these two techniques and
show that the performance of low-precision networks can be significantly
improved by using knowledge distillation techniques. Our approach, Apprentice,
achieves state-of-the-art accuracies using ternary precision and 4-bit
precision for variants of ResNet architecture on ImageNet dataset. We present
three schemes using which one can apply knowledge distillation techniques to
various stages of the train-and-deploy pipeline.

本文介绍了一种组合使用低精度计算和蒸馏知识来提高深度学习网络性能的方法，该方法取得了 ImageNet 数据集上 ResNet 架构各种变体的三元精度和 4 位精度的准确率的最新成果，并提供了三种应用蒸馏知识技术到训练和部署流程中的方案。