Convolution is one of the fundamental operations of deep neural networks with
demanding matrix computation. In a graphic processing unit (GPU), Tensor Core
is a specialized matrix processing hardware equipped with reduced-precision
matrix-multiply-accumulate (MMA) instructions to increase throughput. However,
it is challenging to achieve optimal performance since the best scheduling of
MMA instructions varies for different convolution sizes. In particular,
reduced-precision MMA requires many elements grouped as a matrix operand,
seriously limiting data reuse and imposing packing and layout overhead on the
schedule. This work proposes an automatic scheduling method of
reduced-precision MMA for convolution operation. In this method, we devise a
search space that explores the thread tile and warp sizes to increase the data
reuse despite a large matrix operand of reduced-precision MMA. The search space
also includes options of register-level packing and layout optimization to
lesson overhead of handling reduced-precision data. Finally, we propose a
search algorithm to find the best schedule by learning from the distinctive
candidates. This reduced-precision MMA optimization method is evaluated on
convolution operations of popular neural networks to demonstrate substantial
speedup on Tensor Core compared to the state of the arts with shortened search
time.

这篇论文提出了一种基于 Tensor Core 和 MMA 指令的卷积操作优化方法，该方法通过自动调度探索线程块和 warp 尺寸的搜索空间，包括寄存器级 Packing 和布局优化等选项，使用学习算法找到最佳调度，从而比现有技术在 Tensor Core 上实现了大幅度加速。

从不同的候选者学习，在张量核上优化降低精度卷积程序

Learning from distinctive candidates to optimize reduced-precision convolution program on tensor cores

Over the most recent years, quantized graph neural network (QGNN) attracts
lots of research and industry attention due to its high robustness and low
computation and memory overhead. Unfortunately, the performance gains of QGNN
have never been realized on modern GPU platforms. To this end, we propose the
first Tensor Core (TC) based computing framework, QGTC, to support any-bitwidth
computation for QGNNs on GPUs. We introduce a novel quantized low-bit
arithmetic design based on the low-bit data representation and bit-decomposed
computation. We craft a novel TC-tailored CUDA kernel design by incorporating
3D-stacked bit compression, zero-tile jumping, and non-zero tile reuse
technique to improve the performance systematically. We incorporate an
effective bandwidth-optimized subgraph packing strategy to maximize the
transferring efficiency between CPU host and GPU device. We integrate QGTC with
Pytorch for better programmability and extensibility. Extensive experiments
demonstrate that QGTC achieves on average 2.7x speedup compared with the
state-of-the-art Deep Graph Library framework across diverse settings.

该论文提出了一种基于张量核心的计算框架 QGTC，支持 GPU 上的 QGNN 的任何位宽计算，实验结果表明 QGTC 相对于 Deep Graph Library 框架平均加速了 2.7 倍。

QGTC：使用 GPU 张量核加速量化图神经网络

QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Core

The NVIDIA Volta GPU microarchitecture introduces a specialized unit, called
"Tensor Core" that performs one matrix-multiply-and-accumulate on 4x4 matrices
per clock cycle. The NVIDIA Tesla V100 accelerator, featuring the Volta
microarchitecture, provides 640 Tensor Cores with a theoretical peak
performance of 125 Tflops/s in mixed precision. In this paper, we investigate
current approaches to program NVIDIA Tensor Cores, their performances and the
precision loss due to computation in mixed precision.
Currently, NVIDIA provides three different ways of programming
matrix-multiply-and-accumulate on Tensor Cores: the CUDA Warp Matrix Multiply
Accumulate (WMMA) API, CUTLASS, a templated library based on WMMA, and cuBLAS
GEMM. After experimenting with different approaches, we found that NVIDIA
Tensor Cores can deliver up to 83 Tflops/s in mixed precision on a Tesla V100
GPU, seven and three times the performance in single and half precision
respectively. A WMMA implementation of batched GEMM reaches a performance of 4
Tflops/s. While precision loss due to matrix multiplication with half precision
input might be critical in many HPC applications, it can be considerably
reduced at the cost of increased computation. Our results indicate that HPC
applications using matrix multiplications can strongly benefit from using of
NVIDIA Tensor Cores.

本文介绍了 NVIDIA Volta GPU 微架构中一种特殊的处理器 Tensor Core，它能够在每个时钟周期内对 4x4 矩阵执行一个矩阵乘积累加操作，经过测试，我们发现 NVIDIA Tensor Core 在 Tesla V100 GPU 上能够以混合精度提供 83 Tflops/s 的处理速度，使得 HPC 应用的矩阵乘法能够获得显著的性能提升。