Deep Learning (DL) acceleration support in CPUs has recently gained a lot of
traction, with several companies (Arm, Intel, IBM) announcing products with
specialized matrix engines accessible via GEMM instructions. CPUs are pervasive
and need to handle diverse requirements across DL workloads running in
edge/HPC/cloud platforms. Therefore, as DL workloads embrace sparsity to reduce
the computations and memory size of models, it is also imperative for CPUs to
add support for sparsity to avoid under-utilization of the dense matrix engine
and inefficient usage of the caches and registers. This work presents VEGETA, a
set of ISA and microarchitecture extensions over dense matrix engines to
support flexible structured sparsity for CPUs, enabling programmable support
for diverse DL models with varying degrees of sparsity. Compared to the
state-of-the-art (SOTA) dense matrix engine in CPUs, a VEGETA engine provides
1.09x, 2.20x, 3.74x, and 3.28x speed-ups when running 4:4 (dense), 2:4, 1:4,
and unstructured (95%) sparse DNN layers.

本文介绍了 VEGETA，它是一组指令集和微架构扩展，可支持 CPU 上的灵活结构稀疏性，并与密集矩阵引擎相结合，以便在各种密度的 DL 模型上进行编程，与现有 CPU 中的 SOTA 引擎相比，它提供了 1.09x, 2.20x, 3.74x 和 3.28x 的加速。

VEGETA：CPU 上用于稀疏 / 密集 GEMM Tile 加速的垂直集成扩展

VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs

Rectifier neuron units (ReLUs) have been widely used in deep convolutional
networks. An ReLU converts negative values to zeros, and does not change
positive values, which leads to a high sparsity of neurons. In this work, we
first examine the sparsity of the outputs of ReLUs in some popular deep
convolutional architectures. And then we use the sparsity property of ReLUs to
accelerate the calculation of convolution by skipping calculations of
zero-valued neurons. The proposed sparse convolution algorithm achieves some
speedup improvements on CPUs compared to the traditional matrix-matrix
multiplication algorithm for convolution when the sparsity is not less than
0.9.

本研究针对深度卷积神经网络中常用的整流线性单元（ReLU），探究其输出的稀疏性质，并借此来加速卷积计算，通过跳过计算零值神经元的方法，相比传统矩阵乘法，当稀疏度不低于 0.9 时，在 CPU 上实现了较大的加速。