Quantization has become a popular technique to compress neural networks and
reduce compute cost, but most prior work focuses on studying quantization
without changing the network size. Many real-world applications of neural
networks have compute cost and memory budgets, which can be traded off with
model quality by changing the number of parameters. In this work, we use ResNet
as a case study to systematically investigate the effects of quantization on
inference compute cost-quality tradeoff curves. Our results suggest that for
each bfloat16 ResNet model, there are quantized models with lower cost and
higher accuracy; in other words, the bfloat16 compute cost-quality tradeoff
curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily
quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve
state-of-the-art results on ImageNet for 4-bit ResNet-50 with
quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We
demonstrate the regularizing effect of quantization by measuring the
generalization gap. The quantization method we used is optimized for
practicality: It requires little tuning and is designed with hardware
capabilities in mind. Our work motivates further research into optimal numeric
formats for quantization, as well as the development of machine learning
accelerators supporting these formats. As part of this work, we contribute a
quantization library written in JAX, which is open-sourced at
this https URL

研究表明，使用 4 位和 8 位模型量化的 bfloat16 ResNet 模型计算成本和准确性的权衡曲线优于 bfloat16 模型，其中以 4 位模型量化为主的模型具有最佳 Pareto 曲线，并且基于量化感知训练的 4 位 ResNet-50 模型在 ImageNet 上取得了 77.09％的准确率。