Given a Hyperparameter Optimization(HPO) problem, how to design an algorithm
to find optimal configurations efficiently? Bayesian Optimization(BO) and the
multi-fidelity BO methods employ surrogate models to sample configurations
based on history evaluations. More recent studies obtain better performance by
integrating BO with HyperBand(HB), which accelerates evaluation by early
stopping mechanism. However, these methods ignore the advantage of a suitable
evaluation scheme over the default HyperBand, and the capability of BO is still
constrained by skewed evaluation results. In this paper, we propose FlexHB, a
new method pushing multi-fidelity BO to the limit as well as re-designing a
framework for early stopping with Successive Halving(SH). Comprehensive study
on FlexHB shows that (1) our fine-grained fidelity method considerably enhances
the efficiency of searching optimal configurations, (2) our FlexBand framework
(self-adaptive allocation of SH brackets, and global ranking of configurations
in both current and past SH procedures) grants the algorithm with more
flexibility and improves the anytime performance. Our method achieves superior
efficiency and outperforms other methods on various HPO tasks. Empirical
results demonstrate that FlexHB can achieve up to 6.9X and 11.1X speedups over
the state-of-the-art MFES-HB and BOHB respectively.

通过将 Bayesian Optimization 与 FlexHB 相结合，并重新设计 Successive Halving 的 early stopping 框架，提高了多信度 BO 搜索最优配置的效率，获得了超过其他方法的速度提升。

FlexHB: 一个更高效和灵活的超参数优化框架

FlexHB: a More Efficient and Flexible Framework for Hyperparameter  Optimization

Both computational and experimental material discovery bring forth the
challenge of exploring multidimensional and often non-differentiable parameter
spaces, such as phase diagrams of Hamiltonians with multiple interactions,
composition spaces of combinatorial libraries, processing spaces, and molecular
embedding spaces. Often these systems are expensive or time-consuming to
evaluate a single instance, and hence classical approaches based on exhaustive
grid or random search are too data intensive. This resulted in strong interest
towards active learning methods such as Bayesian optimization (BO) where the
adaptive exploration occurs based on human learning (discovery) objective.
However, classical BO is based on a predefined optimization target, and
policies balancing exploration and exploitation are purely data driven. In
practical settings, the domain expert can pose prior knowledge on the system in
form of partially known physics laws and often varies exploration policies
during the experiment. Here, we explore interactive workflows building on
multi-fidelity BO (MFBO), starting with classical (data-driven) MFBO, then
structured (physics-driven) sMFBO, and extending it to allow human in the loop
interactive iMFBO workflows for adaptive and domain expert aligned exploration.
These approaches are demonstrated over highly non-smooth multi-fidelity
simulation data generated from an Ising model, considering spin-spin
interaction as parameter space, lattice sizes as fidelity spaces, and the
objective as maximizing heat capacity. Detailed analysis and comparison show
the impact of physics knowledge injection and on-the-fly human decisions for
improved exploration, current challenges, and potential opportunities for
algorithm development with combining data, physics and real time human
decisions.

在计算材料发现领域，通过多样的交互工作流程和基于贝叶斯优化的多层次恒定调整，结合数据、物理和实时人工决策，提高了对多维参数空间的探索效率。

加速物理发现的非交互性和交互性多保真贝叶斯优化：当前挑战和未来机遇

Towards accelerating physical discovery via non-interactive and  interactive multi-fidelity Bayesian Optimization: Current challenges and  future opportunities