This paper presents a Spatial Re-parameterization (SpRe) method for the N:M sparsity in CNNs. SpRe is stemmed from an observation regarding the restricted variety in spatial sparsity present in N:M sparsity compared with unstructured sparsity. Particularly, N:M sparsity exhibits a fixed sparsity rate within the spatial domains due to its distinctive pattern that mandates N non-zero components among M successive weights in the input channel dimension of convolution filters. On the contrary, we observe that unstructured sparsity displays a substantial divergence in sparsity across the spatial domains, which we experimentally verified to be very crucial for its robust performance retention compared with N:M sparsity. Therefore, SpRe employs the spatial-sparsity distribution of unstructured sparsity to assign an extra branch in conjunction with the original N:M branch at training time, which allows the N:M sparse network to sustain a similar distribution of spatial sparsity with unstructured sparsity. During inference, the extra branch can be further re-parameterized into the main N:M branch, without exerting any distortion on the sparse pattern or additional computation costs. SpRe has achieved a commendable feat by matching the performance of N:M sparsity methods with state-of-the-art unstructured sparsity methods across various benchmarks. Code and models are anonymously available at \url{https://github.com/zyxxmu/SpRe}.

本文提出了 Spatial Re-parameterization (SpRe) 方法，用于 N:M 稀疏性在 CNN 中的应用。该方法可以在训练期间利用非结构化稀疏性的空间稀疏性分布来为 N:M 稀疏性模型分配一个额外的支路，使其可以维持类似于非结构化稀疏性的空间稀疏性分布。在推理期间，该额外的支路可以进一步重新参数化为主要的 N:M 支路，而不会对稀疏模式或额外的计算成本产生扭曲。SpRe 方法可以在各种基准测试中匹配 N:M 稀疏性方法的表现与最先进的非结构化稀疏性方法。

N:M稀疏性的空间重参数化