One of the main challenges in surrogate modeling is the limited availability
of data due to resource constraints associated with computationally expensive
simulations. Multi-fidelity methods provide a solution by chaining models in a
hierarchy with increasing fidelity, associated with lower error, but increasing
cost. In this paper, we compare different multi-fidelity methods employed in
constructing Gaussian process surrogates for regression. Non-linear
autoregressive methods in the existing literature are primarily confined to
two-fidelity models, and we extend these methods to handle more than two levels
of fidelity. Additionally, we propose enhancements for an existing method
incorporating delay terms by introducing a structured kernel. We demonstrate
the performance of these methods across various academic and real-world
scenarios. Our findings reveal that multi-fidelity methods generally have a
smaller prediction error for the same computational cost as compared to the
single-fidelity method, although their effectiveness varies across different
scenarios.

通过比较多种多保真度方法构建高斯过程代理进行回归，我们发现多保真度方法通常具有较小的预测误差，且对于相同计算成本而言，其效果因不同场景而异。

物理学中回归问题的多准确度高斯过程代理建模

Multi-fidelity Gaussian process surrogate modeling for regression  problems in physics

Multi-fidelity (MF) methods are gaining popularity for enhancing surrogate
modeling and design optimization by incorporating data from various
low-fidelity (LF) models. While most existing MF methods assume a fixed
dataset, adaptive sampling methods that dynamically allocate resources among
fidelity models can achieve higher efficiency in the exploring and exploiting
the design space. However, most existing MF methods rely on the hierarchical
assumption of fidelity levels or fail to capture the intercorrelation between
multiple fidelity levels and utilize it to quantify the value of the future
samples and navigate the adaptive sampling. To address this hurdle, we propose
a framework hinged on a latent embedding for different fidelity models and the
associated pre-posterior analysis to explicitly utilize their correlation for
adaptive sampling. In this framework, each infill sampling iteration includes
two steps: We first identify the location of interest with the greatest
potential improvement using the high-fidelity (HF) model, then we search for
the next sample across all fidelity levels that maximize the improvement per
unit cost at the location identified in the first step. This is made possible
by a single Latent Variable Gaussian Process (LVGP) model that maps different
fidelity models into an interpretable latent space to capture their
correlations without assuming hierarchical fidelity levels. The LVGP enables us
to assess how LF sampling candidates will affect HF response with pre-posterior
analysis and determine the next sample with the best benefit-to-cost ratio.
Through test cases, we demonstrate that the proposed method outperforms the
benchmark methods in both MF global fitting (GF) and Bayesian Optimization (BO)
problems in convergence rate and robustness. Moreover, the method offers the
flexibility to switch between GF and BO by simply changing the acquisition
function.

多层次方法的自适应采样在增强替代建模和设计优化方面具有更高的效率，并且利用潜在嵌入和相关性分析来指导未来样本的选择，从而在全局拟合和贝叶斯优化中提供了更好的性能。

一种非层次性多保真度自适应采样的潜变量方法

A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive  Sampling

Bayesian optimization (BO) is a sequential optimization strategy that is
increasingly employed in a wide range of areas including materials design. In
real world applications, acquiring high-fidelity (HF) data through physical
experiments or HF simulations is the major cost component of BO. To alleviate
this bottleneck, multi-fidelity (MF) methods are used to forgo the sole
reliance on the expensive HF data and reduce the sampling costs by querying
inexpensive low-fidelity (LF) sources whose data are correlated with HF
samples. However, existing multi-fidelity BO (MFBO) methods operate under the
following two assumptions that rarely hold in practical applications: (1) LF
sources provide data that are well correlated with the HF data on a global
scale, and (2) a single random process can model the noise in the fused data.
These assumptions dramatically reduce the performance of MFBO when LF sources
are only locally correlated with the HF source or when the noise variance
varies across the data sources. In this paper, we dispense with these incorrect
assumptions by proposing an MF emulation method that (1) learns a noise model
for each data source, and (2) enables MFBO to leverage highly biased LF sources
which are only locally correlated with the HF source. We illustrate the
performance of our method through analytical examples and engineering problems
on materials design.

基于贝叶斯优化的多重保真度方法的研究，旨在解决在实际应用中存在的假设不准确的问题，其中包括高保真度数据获取成本高和数据源之间相关度局限性的挑战。通过提出一种多重保真度仿真方法，我们可以学习每个数据源的噪声模型，并使多重保真度优化能够利用高度偏差的低保真度数据源，从而提升性能。

关于异构误差对多保真贝叶斯优化的影响

On the Effects of Heterogeneous Errors on Multi-fidelity Bayesian  Optimization

Bandit methods for black-box optimisation, such as Bayesian optimisation, are
used in a variety of applications including hyper-parameter tuning and
experiment design. Recently, \emph{multi-fidelity} methods have garnered
considerable attention since function evaluations have become increasingly
expensive in such applications. Multi-fidelity methods use cheap approximations
to the function of interest to speed up the overall optimisation process.
However, most multi-fidelity methods assume only a finite number of
approximations. In many practical applications however, a continuous spectrum
of approximations might be available. For instance, when tuning an expensive
neural network, one might choose to approximate the cross validation
performance using less data $N$ and/or few training iterations $T$. Here, the
approximations are best viewed as arising out of a continuous two dimensional
space $(N,T)$. In this work, we develop a Bayesian optimisation method, BOCA,
for this setting. We characterise its theoretical properties and show that it
achieves better regret than than strategies which ignore the approximations.
BOCA outperforms several other baselines in synthetic and real experiments.

本文提出了一种基于 Bayesian 优化的多保真度方法 BOCA，并通过理论分析和实验展示了其在神经网络调优等任务上的优越性。