Deep learning models typically use single-precision (FP32) floating point
data types for representing activations and weights, but a slew of recent
research work has shown that computations with reduced-precision data types
(FP16, 16-bit integers, 8-bit integers or even 4- or 2-bit integers) are enough
to achieve same accuracy as FP32 and are much more efficient. Therefore, we
designed fbgemm, a high-performance kernel library, from ground up to perform
high-performance quantized inference on current generation CPUs. fbgemm
achieves efficiency by fusing common quantization operations with a
high-performance gemm implementation and by shape- and size-specific kernel
code generation at runtime. The library has been deployed at Facebook, where it
delivers greater than 2x performance gains with respect to our current
production baseline.

本文介绍了 fbgemm 这一高性能的内核库，其使用了深度学习中的浮点数、16 位整数、8 位整数甚至 4 位或 2 位整数等低精度数据类型，但是仍能保证与高精度数据类型相同的准确性，从而使得量化计算变得更加高效。fbgemm 通过 fusion、gemm 实现和阶段性的内核代码生成，提高计算效率，实践证明在 Facebook 上已经成为了生产基线的 2 倍以上。