Kernel orchestration is the task of mapping the computation defined in
different operators of a deep neural network (DNN) to the execution of GPU
kernels on modern hardware platforms. Prior approaches optimize kernel
orchestration by greedily applying operator fusion, which fuses the computation
of multiple operators into a single kernel, and miss a variety of optimization
opportunities in kernel orchestration.
This paper presents Korch, a tensor program optimizer that discovers optimal
kernel orchestration strategies for tensor programs. Instead of directly fusing
operators, Korch first applies operator fission to decompose tensor operators
into a small set of basic tensor algebra primitives. This decomposition enables
a diversity of fine-grained, inter-operator optimizations. Next, Korch
optimizes kernel orchestration by formalizing it as a constrained optimization
problem, leveraging an off-the-shelf binary linear programming solver to
discover an optimal orchestration strategy, and generating an executable that
can be directly deployed on modern GPU platforms. Evaluation on a variety of
DNNs shows that Korch outperforms existing tensor program optimizers by up to
1.7x on V100 GPUs and up to 1.6x on A100 GPUs. Korch is publicly available at
this https URL

Korch 是一个张量程序优化器，通过将操作符进行分解，然后利用二进制线性规划求解器优化内核编排，生成可在现代 GPU 平台直接部署的可执行文件，从而在 V100 和 A100 GPU 上比现有的张量程序优化器提高了最多 1.7 倍和 1.6 倍的性能。