Hyperparameter optimization (HPO) is concerned with the automated search for
the most appropriate hyperparameter configuration (HPC) of a parameterized
machine learning algorithm. A state-of-the-art HPO method is Hyperband, which,
however, has its own parameters that influence its performance. One of these
parameters, the maximal budget, is especially problematic: If chosen too small,
the budget needs to be increased in hindsight and, as Hyperband is not
incremental by design, the entire algorithm must be re-run. This is not only
costly but also comes with a loss of valuable knowledge already accumulated. In
this paper, we propose incremental variants of Hyperband that eliminate these
drawbacks, and show that these variants satisfy theoretical guarantees
qualitatively similar to those for the original Hyperband with the "right"
budget. Moreover, we demonstrate their practical utility in experiments with
benchmark data sets.

本文提出了超频参数 Hyperband 的增量变体，以消除选择过小的大预算所引起的昂贵成本和有价值已积累知识的流失问题，并在基准数据集实验中展示了它们的实用性和理论保证的类似性。

迭代加深超带

Iterative Deepening Hyperband

Hyperparameter optimization (HPO) is a well-studied research field. However,
the effects and interactions of the components in an HPO pipeline are not yet
well investigated. Then, we ask ourselves: can the landscape of HPO be biased
by the pipeline used to evaluate individual configurations? To address this
question, we proposed to analyze the effect of the HPO pipeline on HPO problems
using fitness landscape analysis. Particularly, we studied the DS-2019 HPO
benchmark data set, looking for patterns that could indicate evaluation
pipeline malfunction, and relate them to HPO performance. Our main findings
are: (i) In most instances, large groups of diverse hyperparameters (i.e.,
multiple configurations) yield the same ill performance, most likely associated
with majority class prediction models; (ii) in these cases, a worsened
correlation between the observed fitness and average fitness in the
neighborhood is observed, potentially making harder the deployment of
local-search based HPO strategies. Finally, we concluded that the HPO pipeline
definition might negatively affect the HPO landscape.

论文探讨 HPO 管道对 HPO 问题的影响，通过适应性景观分析研究了 DS-2019 HPO 基准数据集，发现管道定义可能会对 HPO 景观产生负面影响，并确定了大型多样的高维超参数已经形成了一个模式，可能与大多数类预测模型相关联。

HPO：我们不会再被愚弄

HPO: We won't get fooled again

Meta-learning hyperparameter optimization (HPO) algorithms from prior
experiments is a promising approach to improve optimization efficiency over
objective functions from a similar distribution. However, existing methods are
restricted to learning from experiments sharing the same set of
hyperparameters. In this paper, we introduce the OptFormer, the first
text-based Transformer HPO framework that provides a universal end-to-end
interface for jointly learning policy and function prediction when trained on
vast tuning data from the wild, such as Google's Vizier database, one of the
world's largest HPO datasets. Our extensive experiments demonstrate that the
OptFormer can simultaneously imitate at least 7 different HPO algorithms, which
can be further improved via its function uncertainty estimates. Compared to a
Gaussian Process, the OptFormer also learns a robust prior distribution for
hyperparameter response functions, and can thereby provide more accurate and
better calibrated predictions. This work paves the path to future extensions
for training a Transformer-based model as a general HPO optimizer.

本文介绍了 OptFormer，它是第一个基于文本的 Transformer HPO 框架，可以在从 Google 的 Vizier 数据库等多种调整数据中训练，以提供学习策略和功能预测的通用端到端接口。OptFormer 能同时模拟至少 7 种不同的 HPO 算法，可以通过其函数不确定性估计进一步改进，并学习到对超参数响应函数的强健先验分布，可以提供更准确和更好的校准预测，这项工作为训练基于 Transformer 模型作为通用 HPO 优化器的未来扩展铺平了道路。