Mixed-precision quantization of efficient networks often suffer from
activation instability encountered in the exploration of bit selections. To
address this problem, we propose a novel method called MetaMix which consists
of bit selection and weight training phases. The bit selection phase iterates
two steps, (1) the mixed-precision-aware weight update, and (2) the bit-search
training with the fixed mixed-precision-aware weights, both of which combined
reduce activation instability in mixed-precision quantization and contribute to
fast and high-quality bit selection. The weight training phase exploits the
weights and step sizes trained in the bit selection phase and fine-tunes them
thereby offering fast training. Our experiments with efficient and
hard-to-quantize networks, i.e., MobileNet v2 and v3, and ResNet-18 on ImageNet
show that our proposed method pushes the boundary of mixed-precision
quantization, in terms of accuracy vs. operations, by outperforming both mixed-
and single-precision SOTA methods.

提出了一种名为 MetaMix 的新方法，通过位选择和权重训练阶段来解决混合精度量化中的激活不稳定性问题，其通过降低混合精度量化中的激活不稳定性，实现快速、高质量的位选择，而权重训练阶段则利用位选择阶段中训练的权重和步长进行微调，从而提供快速训练。在 ImageNet 数据集上对 MobileNet v2、v3 和 ResNet-18 等高效且难以量化的网络进行的实验显示，我们的方法在准确度与操作之间推动了混合精度量化的发展边界，并且在性能上超过了混合精度和单精度的 SOTA 方法。

MetaMix: 混合精度激活量化的元状态精确搜索器

MetaMix: Meta-state Precision Searcher for Mixed-precision Activation  Quantization

4-bit and lower precision mobile models are required due to the
ever-increasing demand for better energy efficiency in mobile devices. In this
work, we report that the activation instability induced by weight quantization
(AIWQ) is the key obstacle to sub-4-bit quantization of mobile networks. To
alleviate the AIWQ problem, we propose a novel training method called
PROgressive-Freezing Iterative Training (PROFIT), which attempts to freeze
layers whose weights are affected by the instability problem stronger than the
other layers. We also propose a differentiable and unified quantization method
(DuQ) and a negative padding idea to support asymmetric activation functions
such as h-swish. We evaluate the proposed methods by quantizing MobileNet-v1,
v2, and v3 on ImageNet and report that 4-bit quantization offers comparable
(within 1.48 % top-1 accuracy) accuracy to full precision baseline. In the
ablation study of the 3-bit quantization of MobileNet-v3, our proposed method
outperforms the state-of-the-art method by a large margin, 12.86 % of top-1
accuracy.

本文提出了 PROFIT 训练法和 DuQ 量化方法，用于解决低位精度移动网络中的权重量化引起的激活不稳定性问题，以提高移动设备的能效，并在 ImageNet 数据集上评估验证，与全精度基线相比，4 位量化提供了相似的准确性。在对 MobileNet-v3 进行的 3 位量化剖析研究中，本文提出的方法的 top-1 准确率显著优于现有的方法。