This paper studies using foundational large language models (LLMs) to make
decisions during hyperparameter optimization (HPO). Empirical evaluations
demonstrate that in settings with constrained search budgets, LLMs can perform
comparably or better than traditional HPO methods like random search and
Bayesian optimization on standard benchmarks. Furthermore, we propose to treat
the code specifying our model as a hyperparameter, which the LLM outputs, going
beyond the capabilities of existing HPO approaches. Our findings suggest that
LLMs are a promising tool for improving efficiency in the traditional
decision-making problem of hyperparameter optimization.

使用大型语言模型（LLMs）在超参数优化（HPO）期间进行决策，在有限搜索预算的情况下，通过实证评估发现，LLMs 在标准基准测试上可以表现与传统 HPO 方法（如随机搜索和贝叶斯优化）相当甚至更好。此外，我们提出将定义模型的代码作为超参数进行处理，LLMs 输出该超参数，超越了现有 HPO 方法的能力。我们的研究结果表明，LLMs 是提高传统超参数优化中效率的一种有前景的工具。