As diverse high-performance computing (HPC) systems are built, many
opportunities arise for applications to solve larger problems than ever before.
Given the significantly increased complexity of these HPC systems and
application tuning, empirical performance tuning, such as autotuning, has
emerged as a promising approach in recent years. Despite its effectiveness,
autotuning is often a computationally expensive approach. Transfer learning
(TL)-based autotuning seeks to address this issue by leveraging the data from
prior tuning. Current TL methods for autotuning spend significant time modeling
the relationship between parameter configurations and performance, which is
ineffective for few-shot (that is, few empirical evaluations) tuning on new
tasks. We introduce the first generative TL-based autotuning approach based on
the Gaussian copula (GC) to model the high-performing regions of the search
space from prior data and then generate high-performing configurations for new
tasks. This allows a sampling-based approach that maximizes few-shot
performance and provides the first probabilistic estimation of the few-shot
budget for effective TL-based autotuning. We compare our generative TL approach
with state-of-the-art autotuning techniques on several benchmarks. We find that
the GC is capable of achieving 64.37% of peak few-shot performance in its first
evaluation. Furthermore, the GC model can determine a few-shot transfer budget
that yields up to 33.39$\times$ speedup, a dramatic improvement over the
20.58$\times$ speedup using prior techniques.

给定的研究论文的主要课题和研究领域包括高性能计算、自动调优、迁移学习、小样本性能和高斯构造模型。

基于转移学习的高斯 Copula 自动调谐

Transfer-Learning-Based Autotuning Using Gaussian Copula

Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be
effective in handling high-dimensional computational problems, providing
high-quality empirical performance as well as strong probabilistic guarantees.
However, their practical application is complicated by the fact that the user
needs to set various algorithm-specific tuning parameters which are different
than those used in traditional NLA. This paper demonstrates how a
surrogate-based autotuning approach can be used to address fundamental problems
of parameter selection in RandNLA algorithms. In particular, we provide a
detailed investigation of surrogate-based autotuning for
sketch-and-precondition (SAP) based randomized least squares methods, which
have been one of the great success stories in modern RandNLA. Empirical results
show that our surrogate-based autotuning approach can achieve near-optimal
performance with much less tuning cost than a random search (up to about 4x
fewer trials of different parameter configurations). Moreover, while our
experiments focus on least squares, our results demonstrate a general-purpose
autotuning pipeline applicable to any kind of RandNLA algorithm.

该论文展示了如何使用基于代理模型的自动调整方法来解决 RandNLA 算法中参数选择的基本问题，并详细研究了基于代理模型的自动调整方法在基于随机预处理的最小二乘法中的应用。实验结果表明，我们的基于代理模型的自动调整方法可以以较少的调整成本实现接近最优的性能，同时适用于任何类型的 RandNLA 算法。