Quantization has become a mainstream compression technique for reducing model
size, computational requirements, and energy consumption for modern deep neural
networks (DNNs). With the improved numerical support in recent hardware,
including multiple variants of integer and floating point, mixed-precision
quantization has become necessary to achieve high-quality results with low
model cost. Prior mixed-precision quantization methods have performed a
post-training quantization search, which compromises on accuracy, or a
differentiable quantization search, which leads to high memory usage from
branching. Therefore, we propose the first one-shot mixed-precision
quantization search that eliminates the need for retraining in both integer and
low-precision floating point models. We evaluate our floating-point and integer
quantization search (FLIQS) on multiple convolutional networks and vision
transformer models to discover Pareto-optimal models. Our approach discovers
models that improve upon uniform precision, manual mixed-precision, and recent
integer quantization search methods. With the proposed integer quantization
search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and
ResNet-50 by 0.90% points with equivalent model cost over previous methods.
Additionally, for the first time, we explore a novel mixed-precision
floating-point search and improve MobileNetV2 by up to 0.98% points compared to
prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously
search a joint quantization and neural architecture space and improve the
ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2
search space.

量化已成为现代深度神经网络的主流压缩技术，本文提出了一种一次性的混合精度量化搜索方法，通过整数和低精度浮点模型在不需要重新训练的情况下提高了 ResNet-18 和 ResNet-50 在 ImageNet 上的准确性，并探索了混合精度浮点搜索和量化与神经架构空间的联合搜索。

FLIQS: 单次混合精度浮点数与整数量化搜索

FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization  Search

Matrix-matrix multiplication is a key computational kernel for numerous
applications in science and engineering, with ample parallelism and data
locality that lends itself well to high-performance implementations. Many
matrix multiplication-dependent applications can use reduced-precision integer
or fixed-point representations to increase their performance and energy
efficiency while still offering adequate quality of results. However, precision
requirements may vary between different application phases or depend on input
data, rendering constant-precision solutions ineffective. BISMO, a vectorized
bit-serial matrix multiplication overlay for reconfigurable computing,
previously utilized the excellent binary-operation performance of FPGAs to
offer a matrix multiplication performance that scales with required precision
and parallelism. We show how BISMO can be scaled up on Xilinx FPGAs using an
arithmetic architecture that better utilizes 6-LUTs. The improved BISMO
achieves a peak performance of 15.4 binary TOPS on the Ultra96 board with a
Xilinx UltraScale+ MPSoC.

本文介绍了一种基于 FPGAs 的向量化位串行矩阵乘法叠加器，通过使用更好地利用 6-LUT 的算术结构，可以将该矩阵乘法的性能提高到 15.4 个二进制 TOPS。