Quantizing neural networks is one of the most effective methods for achieving
efficient inference on mobile and embedded devices. In particular, mixed
precision quantized (MPQ) networks, whose layers can be quantized to different
bitwidths, achieve better task performance for the same resource constraint
compared to networks with homogeneous bitwidths. However, finding the optimal
bitwidth allocation is a challenging problem as the search space grows
exponentially with the number of layers in the network. In this paper, we
propose QBitOpt, a novel algorithm for updating bitwidths during
quantization-aware training (QAT). We formulate the bitwidth allocation problem
as a constraint optimization problem. By combining fast-to-compute
sensitivities with efficient solvers during QAT, QBitOpt can produce
mixed-precision networks with high task performance guaranteed to satisfy
strict resource constraints. This contrasts with existing mixed-precision
methods that learn bitwidths using gradients and cannot provide such
guarantees. We evaluate QBitOpt on ImageNet and confirm that we outperform
existing fixed and mixed-precision methods under average bitwidth constraints
commonly found in the literature.

提出了一种名为 QBitOpt 的算法，通过量化感知训练（QAT）期间更新比特宽度，将位宽分配问题转化为约束优化问题，利用快速计算的灵敏度和高效求解器，生成满足严格资源约束的高性能任务的混合精度网络。在常见的位宽约束下，在 ImageNet 上评价 QBitOpt 并证实了在固定和混合精度方法中的优越性。

QBitOpt：训练过程中快速准确的位宽重新分配

QBitOpt: Fast and Accurate Bitwidth Reallocation during Training

When quantizing neural networks, assigning each floating-point weight to its
nearest fixed-point value is the predominant approach. We find that, perhaps
surprisingly, this is not the best we can do. In this paper, we propose
AdaRound, a better weight-rounding mechanism for post-training quantization
that adapts to the data and the task loss. AdaRound is fast, does not require
fine-tuning of the network, and only uses a small amount of unlabelled data. We
start by theoretically analyzing the rounding problem for a pre-trained neural
network. By approximating the task loss with a Taylor series expansion, the
rounding task is posed as a quadratic unconstrained binary optimization
problem. We simplify this to a layer-wise local loss and propose to optimize
this loss with a soft relaxation. AdaRound not only outperforms
rounding-to-nearest by a significant margin but also establishes a new
state-of-the-art for post-training quantization on several networks and tasks.
Without fine-tuning, we can quantize the weights of Resnet18 and Resnet50 to 4
bits while staying within an accuracy loss of 1%.

本文提出了 AdaRound，它是一种更好的后训练量化权重舍入机制，能够适应数据和任务损失，不需要对网络进行微调，并且只使用少量无标签数据。