While neural networks have advanced the frontiers in many machine learning
applications, they often come at a high computational cost. Reducing the power
and latency of neural network inference is vital to integrating modern networks
into edge devices with strict power and compute requirements. Neural network
quantization is one of the most effective ways of achieving these savings, but
the additional noise it induces can lead to accuracy degradation. In this white
paper, we present an overview of neural network quantization using AI Model
Efficiency Toolkit (AIMET). AIMET is a library of state-of-the-art quantization
and compression algorithms designed to ease the effort required for model
optimization and thus drive the broader AI ecosystem towards low latency and
energy-efficient inference. AIMET provides users with the ability to simulate
as well as optimize PyTorch and TensorFlow models. Specifically for
quantization, AIMET includes various post-training quantization (PTQ, cf.
chapter 4) and quantization-aware training (QAT, cf. chapter 5) techniques that
guarantee near floating-point accuracy for 8-bit fixed-point inference. We
provide a practical guide to quantization via AIMET by covering PTQ and QAT
workflows, code examples and practical tips that enable users to efficiently
and effectively quantize models using AIMET and reap the benefits of low-bit
integer inference.

本文介绍了使用 AI Model Efficiency Toolkit (AIMET) 进行神经网络量化的概述，其中 AIMET 包括多种基于 PyTorch 和 TensorFlow 的后训练量化和量化感知训练技术，可保证 8 位定点推理的接近浮点精度，并提供关于 PTQ 和 QAT 工作流程，代码示例和实用技巧的实用指南，以便用户有效地量化模型并获得低位整数推理的好处。