Quantization for CNN has shown significant progress with the intention of
reducing the cost of computation and storage with low-bitwidth data
representations. There are, however, no systematic studies on how an existing
full-bitwidth processing unit, such as ALU in CPUs and DSP in FPGAs, can be
better utilized to deliver significantly higher computation throughput for
convolution under various quantized bitwidths. In this study, we propose
HiKonv, a unified solution that maximizes the throughput of convolution on a
given underlying processing unit with low-bitwidth quantized data inputs
through novel bit-wise management and parallel computation. We establish
theoretical framework and performance models using a full-bitwidth multiplier
for highly parallelized low-bitwidth convolution, and demonstrate new
breakthroughs for high-performance computing in this critical domain. For
example, a single 32-bit processing unit in CPU can deliver 128 binarized
convolution operations (multiplications and additions) and 13 4-bit convolution
operations with a single multiplication instruction, and a single 27x18
multiplier in the FPGA DSP can deliver 60, 8 or 2 convolution operations with
1, 4 or 8-bit inputs in one clock cycle. We demonstrate the effectiveness of
HiKonv on both CPU and FPGA. On CPU, HiKonv outperforms the baseline
implementation with 1 to 8-bit inputs and provides up to 7.6x and 1.4x
performance improvements for 1-D convolution, and performs 2.74x and 3.19x over
the baseline implementation for 4-bit signed and unsigned data inputs for 2-D
convolution. On FPGA, HiKonv solution enables a single DSP to process multiple
convolutions with a shorter processing latency. For binarized input, each DSP
with HiKonv is equivalent up to 76.6 LUTs. Compared to the DAC-SDC 2020
champion model, HiKonv achieves a 2.37x throughput improvement and 2.61x DSP
efficiency improvement, respectively.

提出了 HiKonv，一种针对特定底层处理单元和低位宽量化数据输入最大化卷积吞吐量的统一解决方案，使用全位宽乘积器建立理论框架和性能模型，可在 CPU 和 FPGA 上实现卓越表现。

HiKonv：通过新的按位管理和计算最大化量化卷积的吞吐量

HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation

Neural network quantization is an effective way to compress deep models and
improve their execution latency and energy efficiency, so that they can be
deployed on mobile or embedded devices. Existing quantization methods require
original data for calibration or fine-tuning to get better performance.
However, in many real-world scenarios, the data may not be available due to
confidential or private issues, thereby making existing quantization methods
not applicable. Moreover, due to the absence of original data, the recently
developed generative adversarial networks (GANs) cannot be applied to generate
data. Although the full-precision model may contain rich data information, such
information alone is hard to exploit for recovering the original data or
generating new meaningful data. In this paper, we investigate a
simple-yet-effective method called Generative Low-bitwidth Data Free
Quantization (GDFQ) to remove the data dependence burden. Specifically, we
propose a knowledge matching generator to produce meaningful fake data by
exploiting classification boundary knowledge and distribution information in
the pre-trained model. With the help of generated data, we can quantize a model
by learning knowledge from the pre-trained model. Extensive experiments on
three data sets demonstrate the effectiveness of our method. More critically,
our method achieves much higher accuracy on 4-bit quantization than the
existing data free quantization method. Code is available at
this https URL

本文提出了一种简单而有效的方法，叫做生成式低比特数据无关量化 (GDFQ)，通过使用预训练模型中的分类边界知识和分布信息，利用知识匹配生成器生成有意义的虚假数据，从而消除数据依赖负担，来量化模型。在三个数据集上的广泛实验证明了本方法的有效性，并且在 4 位量化上的精确性要比现有的数据无关量化方法高得多。