In recent years, there has been a flurry of research in deep neural network pruning and compression. Early approaches prune weights individually. However, it is difficult to take advantage of the resulting unstructured sparsity patterns on modern hardware like GPUs. As a result, pruning strategies which impose sparsity structures in the weights have become more popular. However,these structured pruning approaches typically lead to higher losses in accuracy than unstructured pruning. In this paper, we present SparseRT, a code generator that leverage unstructured sparsity to accelerate sparse linear algebra operations in deep learning inference on GPUs. For 1x1 convolutions and fully connected layers, we demonstrate geometric mean of speedups of 3.4x over the equivalent dense computation at 90% sparsity and 5.4x at 95% sparsity when evaluated on hundreds of test cases in deep learning. For sparse 3x3 convolutions, we show speedups of over 5x on use cases in ResNet-50.

本文介绍了一种基于 unstructured sparsity 的 code generator - SparseRT，通过对于1x1 convolutions 以及 fully connected layers 进行计算，该方法相对于 equivalent dense computation 速度提升了 3.4x 至 5.4x，同时处理 ResNet-50 的 sparse 3x3 convolutions 取得了 5x 的速度提升，能够有效地加速 GPU 上的深度学习操作。

SparseRT: 基于GPU加速的深度学习推断非结构化稀疏矩阵运算