GPU kernels have come to the forefront of comput- ing due to their utility in
varied fields, from high-performance computing to machine learning. A typical
GPU compute kernel is invoked millions, if not billions of times in a typical
application, which makes their performance highly critical. Due to the unknown
nature of the optimization surface, an exhaustive search is required to
discover the global optimum, which is infeasible due to the possible
exponential number of parameter combinations. In this work, we propose a
methodology that uses deep sequence- to-sequence models to predict the optimal
tuning parameters governing compute kernels. This work considers the prediction
of kernel parameters as a sequence to the sequence translation problem,
borrowing models from the Natural Language Process- ing (NLP) domain.
Parameters describing the input, output and weight tensors are considered as
the input language to the model that emits the corresponding kernel parameters.
In essence, the model translates the problem parameter language to kernel
parameter language. The core contributions of this work are: a) Proposing that
a sequence to sequence model can accurately learn the performance dynamics of a
GPU compute kernel b) A novel network architecture which predicts the kernel
tuning parameters for GPU kernels, c) A constrained beam search which
incorporates the physical limits of the GPU hardware as well as other expert
knowledge reducing the search space. The proposed algorithm can achieve more
than 90% accuracy on various convolutional kernels in MIOpen, the AMD machine
learning primitives library. As a result, the proposed technique can reduce the
development time and compute resources required to tune unseen input
configurations, resulting in shorter development cycles, reduced development
costs, and better user experience.

使用深度序列到序列模型预测 GPU 计算内核的最佳调优参数，能够准确学习 GPU 计算内核的性能动态，减少开发时间和计算资源。

使用深度序列模型预测最佳核调参

Optimal Kernel Tuning Parameter Prediction using Deep Sequence Models

In the realm of competitive sports, understanding the performance dynamics of
athletes, represented by the age curve (showing progression, peak, and
decline), is vital. Our research introduces a novel framework for quantifying
age-specific treatment effects, enhancing the granularity of performance
trajectory analysis. Firstly, we propose a methodology for estimating the age
curve using game-level data, diverging from traditional season-level data
approaches, and tackling its inherent complexities with a meta-learner
framework that leverages advanced machine learning models. This approach
uncovers intricate non-linear patterns missed by existing methods. Secondly,
our framework enables the identification of causal effects, allowing for a
detailed examination of age curves under various conditions. By defining the
Age-Conditioned Treatment Effect (ACTE), we facilitate the exploration of
causal relationships regarding treatment impacts at specific ages. Finally,
applying this methodology to study the effects of rest days on performance
metrics, particularly across different ages, offers valuable insights into load
management strategies' effectiveness. Our findings underscore the importance of
tailored rest periods, highlighting their positive impact on athlete
performance and suggesting a reevaluation of current management practices for
optimizing athlete performance.

通过引入新的分析框架，本研究对运动员的表现动态进行了量化，提高了对性能轨迹的细致分析，揭示了隐含的非线性模式，并通过定义年龄条件化治疗效应（ACTE）的方式，探索了年龄对治疗效果的因果关系，最终发现个性化休息周期对运动员性能的积极影响，为优化运动员表现的管理实践提供了有价值的见解。