Hyperparameter optimization (HPO) and neural architecture search (NAS) are powerful in attaining state-of-the-art machine learning models, with Bayesian optimization (BO) standing out as a mainstream method. Extending BO into the multi-fidelity setting has been an emerging research topic, but faces the challenge of determining an appropriate fidelity for each hyperparameter configuration to fit the surrogate model. To tackle the challenge, we propose a multi-fidelity BO method named FastBO, which adaptively decides the fidelity for each configuration and efficiently offers strong performance. The advantages are achieved based on the novel concepts of efficient point and saturation point for each configuration.We also show that our adaptive fidelity identification strategy provides a way to extend any single-fidelity method to the multi-fidelity setting, highlighting its generality and applicability.

本研究解决了在超参数优化（HPO）和神经架构搜索（NAS）中，如何为每个超参数配置确定适当的保真度这一问题。提出的FastBO方法通过自适应选择配置的保真度，并利用有效点和饱和点的概念，显著提高了性能。这种自适应保真度识别策略还展示了其将任何单一保真度方法扩展到多保真度设置的通用性和适用性。

FastBO：基于自适应保真度识别的快速超参数优化和神经架构搜索