Quantization techniques can reduce the size of Deep Neural Networks and
improve inference latency and throughput by taking advantage of high throughput
integer instructions. In this paper we review the mathematical aspects of
quantization parameters and evaluate their choices on a wide range of neural
network models for different application domains, including vision, speech, and
language. We focus on quantization techniques that are amenable to acceleration
by processors with high-throughput integer math pipelines. We also present a
workflow for 8-bit quantization that is able to maintain accuracy within 1% of
the floating-point baseline on all networks studied, including models that are
more difficult to quantize, such as MobileNets and BERT-large.

本文介绍了量化技术如何减小深度神经网络的规模，提高推理延迟和吞吐量，并评估它们在各种应用领域的不同神经网络模型上的选择，包括视觉、语音和语言等方面，并重点介绍适用于高吞吐量整数数学流水线处理器加速的量化技术。同时，还提供了一种 8 位量化工作流，能够在所有研究的网络上保持 1% 的浮点基线精度，包括更难量化的模型，如 MobileNets 和 BERT-large。