Unstructured pruning reduces the memory footprint in deep neural networks
(DNNs). Recently, researchers proposed different types of structural pruning
intending to reduce also the computation complexity. In this work, we first
suggest a new measure called mask-diversity which correlates with the expected
accuracy of the different types of structural pruning. We focus on the recently
suggested N:M fine-grained block sparsity mask, in which for each block of M
weights, we have at least N zeros. While N:M fine-grained block sparsity allows
acceleration in actual modern hardware, it can be used only to accelerate the
inference phase. In order to allow for similar accelerations in the training
phase, we suggest a novel transposable fine-grained sparsity mask, where the
same mask can be used for both forward and backward passes. Our transposable
mask guarantees that both the weight matrix and its transpose follow the same
sparsity pattern; thus, the matrix multiplication required for passing the
error backward can also be accelerated. We formulate the problem of finding the
optimal transposable-mask as a minimum-cost flow problem. Additionally, to
speed up the minimum-cost flow computation, we also introduce a fast
linear-time approximation that can be used when the masks dynamically change
during training. Our experiments suggest a 2x speed-up in the matrix
multiplications with no accuracy degradation over vision and language models.
Finally, to solve the problem of switching between different structure
constraints, we suggest a method to convert a pre-trained model with
unstructured sparsity to an N:M fine-grained block sparsity model with little
to no training. A reference implementation can be found at
this https URL

本研究提出了一种新的称为掩码多样性的度量方法来降低 DNN 的内存占用，并介绍了一种可同时用于前向和后向传递的新型可转置细粒度稀疏掩码，通过使用最小成本流问题来寻找最优的可转置掩码，并且还提出了一种将无结构稀疏模型转换为 N:M 细粒度块稀疏模型的方法。