Recent machine learning methods use increasingly large deep neural networks
to achieve state of the art results in various tasks. The gains in performance
come at the cost of a substantial increase in computation and storage
requirements. This makes real-time implementations on limited resources
hardware a challenging task. One popular approach to address this challenge is
to perform low-bit precision computations via neural network quantization.
However, aggressive quantization generally entails a severe penalty in terms of
accuracy, and often requires retraining of the network, or resorting to higher
bit precision quantization. In this paper, we formalize the linear quantization
task as a Minimum Mean Squared Error (MMSE) problem for both weights and
activations, allowing low-bit precision inference without the need for full
network retraining. The main contributions of our approach are the
optimizations of the constrained MSE problem at each layer of the network, the
hardware aware partitioning of the network parameters, and the use of multiple
low precision quantized tensors for poorly approximated layers. The proposed
approach allows 4 bits integer (INT4) quantization for deployment of pretrained
models on limited hardware resources. Multiple experiments on various network
architectures show that the suggested method yields state of the art results
with minimal loss of tasks accuracy.

本文提出了一种在有限硬件资源上实现预训练模型的 4 位整数（INT4）量化的优化方法，将线性量化任务形式化为最小均方误差（MMSE）问题，并对网络的每一层进行限制 MSE 问题的优化以及多个量化张量的硬件感知分区，除少量精度降低外，在多种网络架构上实现最先进的结果。