Gaussian Process Regression (GPR) is widely used in statistics and machine
learning for prediction tasks requiring uncertainty measures. Its efficacy
depends on the appropriate specification of the mean function, covariance
kernel function, and associated hyperparameters. Severe misspecifications can
lead to inaccurate results and problematic consequences, especially in
safety-critical applications. However, a systematic approach to handle these
misspecifications is lacking in the literature. In this work, we propose a
general framework to address these issues. Firstly, we introduce a flexible
two-stage GPR framework that separates mean prediction and uncertainty
quantification (UQ) to prevent mean misspecification, which can introduce bias
into the model. Secondly, kernel function misspecification is addressed through
a novel automatic kernel search algorithm, supported by theoretical analysis,
that selects the optimal kernel from a candidate set. Additionally, we propose
a subsampling-based warm-start strategy for hyperparameter initialization to
improve efficiency and avoid hyperparameter misspecification. With much lower
computational cost, our subsampling-based strategy can yield competitive or
better performance than training exclusively on the full dataset. Combining all
these components, we recommend two GPR methods-exact and scalable-designed to
match available computational resources and specific UQ requirements. Extensive
evaluation on real-world datasets, including UCI benchmarks and a
safety-critical medical case study, demonstrates the robustness and precision
of our methods.

通过引入灵活的两阶段高斯过程回归框架、自动核搜索算法、子采样启动策略和精确与可伸缩两种方法，本研究提出了一种泛用框架来解决 GPR 中的表示偏差、核函数错误和超参数错误问题，并在真实世界数据集上进行了广泛评估，验证了方法的鲁棒性和精确性。

高效的双阶高斯过程回归 —— 自动核搜索和子采样

Efficient Two-Stage Gaussian Process Regression Via Automatic Kernel  Search and Subsampling

Automating statistical modelling is a challenging problem in artificial
intelligence. The Automatic Statistician takes a first step in this direction,
by employing a kernel search algorithm with Gaussian Processes (GP) to provide
interpretable statistical models for regression problems. However this does not
scale due to its $O(N^3)$ running time for the model selection. We propose
Scalable Kernel Composition (SKC), a scalable kernel search algorithm that
extends the Automatic Statistician to bigger data sets. In doing so, we derive
a cheap upper bound on the GP marginal likelihood that sandwiches the marginal
likelihood with the variational lower bound . We show that the upper bound is
significantly tighter than the lower bound and thus useful for model selection.

SKC 是一种可扩展的核搜索算法，它利用 Gaussian Processes 为回归问题提供可解释的统计模型，并提出一种廉价的上界，加速模型选择。