In this paper, we address the problem of cost-sensitive multi-fidelity
Bayesian Optimization (BO) for efficient hyperparameter optimization (HPO).
Specifically, we assume a scenario where users want to early-stop the BO when
the performance improvement is not satisfactory with respect to the required
computational cost. Motivated by this scenario, we introduce utility, which is
a function predefined by each user and describes the trade-off between cost and
performance of BO. This utility function, combined with our novel acquisition
function and stopping criterion, allows us to dynamically choose for each BO
step the best configuration that we expect to maximally improve the utility in
future, and also automatically stop the BO around the maximum utility. Further,
we improve the sample efficiency of existing learning curve (LC) extrapolation
methods with transfer learning, while successfully capturing the correlations
between different configurations to develop a sensible surrogate function for
multi-fidelity BO. We validate our algorithm on various LC datasets and found
it outperform all the previous multi-fidelity BO and transfer-BO baselines we
consider, achieving significantly better trade-off between cost and performance
of BO.

本文研究了基于成本敏感的多保真贝叶斯优化算法用于高效的超参数优化问题，引入了用户预定义的效用函数来描述成本和性能之间的权衡，并提出了一种新的获取函数和停止准则以动态选择每个步骤的最佳配置，并最大程度地提高未来效用，同时自动终止优化过程，通过转移学习提高学习曲线外推方法的样本效率，能够捕捉不同配置之间的相关性，为多保真贝叶斯优化提供合理的代理函数，并在各种学习曲线数据集上验证算法，优于现有基线方法，实现更好的成本和性能的权衡。