Deep learning on point clouds has received increased attention thanks to its
wide applications in AR/VR and autonomous driving. These applications require
low latency and high accuracy to provide real-time user experience and ensure
user safety. Unlike conventional dense workloads, the sparse and irregular
nature of point clouds poses severe challenges to running sparse CNNs
efficiently on the general-purpose hardware. Furthermore, existing sparse
acceleration techniques for 2D images do not translate to 3D point clouds. In
this paper, we introduce TorchSparse, a high-performance point cloud inference
engine that accelerates the sparse convolution computation on GPUs. TorchSparse
directly optimizes the two bottlenecks of sparse convolution: irregular
computation and data movement. It applies adaptive matrix multiplication
grouping to trade computation for better regularity, achieving 1.4-1.5x speedup
for matrix multiplication. It also optimizes the data movement by adopting
vectorized, quantized and fused locality-aware memory access, reducing the
memory movement cost by 2.7x. Evaluated on seven representative models across
three benchmark datasets, TorchSparse achieves 1.6x and 1.5x measured
end-to-end speedup over the state-of-the-art MinkowskiEngine and SpConv,
respectively.

本文介绍 TorchSparse，一种高效的基于 GPU 加速的稀疏卷积计算引擎，应用于 AR/VR、自动驾驶等方面。通过采用自适应矩阵乘法分组技术和集成向量化、量化及融合的本地化优化存储访问技术，实现了 1.4-1.5 倍的速度提升和 2.7 倍的内存移动代价降低，相较当前最优方法 MinkowskiEngine 和 SpConv，分别实现了 1.6 倍和 1.5 倍的整体加速效果。