In this paper, we critically examine the prevalent practice of using additive
mixtures of Mat\'ern kernels in single-output Gaussian process (GP) models and
explore the properties of multiplicative mixtures of Mat\'ern kernels for
multi-output GP models. For the single-output case, we derive a series of
theoretical results showing that the smoothness of a mixture of Mat\'ern
kernels is determined by the least smooth component and that a GP with such a
kernel is effectively equivalent to the least smooth kernel component.
Furthermore, we demonstrate that none of the mixing weights or parameters
within individual kernel components are identifiable. We then turn our
attention to multi-output GP models and analyze the identifiability of the
covariance matrix $A$ in the multiplicative kernel $K(x,y) = AK_0(x,y)$, where
$K_0$ is a standard single output kernel such as Mat\'ern. We show that $A$ is
identifiable up to a multiplicative constant, suggesting that multiplicative
mixtures are well suited for multi-output tasks. Our findings are supported by
extensive simulations and real applications for both single- and multi-output
settings. This work provides insight into kernel selection and interpretation
for GP models, emphasizing the importance of choosing appropriate kernel
structures for different tasks.

本文对使用 Matern 核的加法混合在单输出高斯过程模型中的常见做法进行了批判性分析，并研究了 Matern 核的乘法混合在多输出高斯过程模型中的特性。通过一系列理论结果，我们推导出对于单输出情况，Matern 核的平滑性由最不平滑的成分决定，并且具有这样一个内核的高斯过程等效于最不平滑的核成分。此外，我们证明了各个核组件中的混合权重或参数都不可辨。随后，我们转向多输出高斯过程模型，并分析了乘法核中协方差矩阵 A 的可辨识性，其中 K (x, y) = AK_0 (x, y) ，K_0 是标准的单输出核，比如 Matern 核。我们展示了 A 可辨识，但具有多个倍数常数的特性，这表明乘法混合非常适用于多输出任务。我们的发现得到了对单输出和多输出情况的大量模拟和实际应用的支持。这项研究为高斯过程模型的核选择和解释提供了洞察，强调了在不同任务中选择适当的核结构的重要性。

高斯过程模型的可辨识性和可解释性研究

On the Identifiability and Interpretability of Gaussian Process Models

Prior beliefs about the latent function to shape inductive biases can be
incorporated into a Gaussian Process (GP) via the kernel. However, beyond
kernel choices, the decision-making process of GP models remains poorly
understood. In this work, we contribute an analysis of the loss landscape for
GP models using methods from physics. We demonstrate $\nu$-continuity for
Matern kernels and outline aspects of catastrophe theory at critical points in
the loss landscape. By directly including $\nu$ in the hyperparameter
optimisation for Matern kernels, we find that typical values of $\nu$ are far
from optimal in terms of performance, yet prevail in the literature due to the
increased computational speed. We also provide an a priori method for
evaluating the effect of GP ensembles and discuss various voting approaches
based on physical properties of the loss landscape. The utility of these
approaches is demonstrated for various synthetic and real datasets. Our
findings provide an enhanced understanding of the decision-making process
behind GPs and offer practical guidance for improving their performance and
interpretability in a range of applications.

本文研究了高斯过程模型中决策过程的损失景观，着重探讨了 Matern 核函数、紧急点的相关性质以及 $
u$ 的超参数优化问题，为 GP 模型的解释性和性能提升提供了实用的指导。

物理启发式方法探索理解高斯过程

Physics Inspired Approaches Towards Understanding Gaussian Processes

This work brings together two powerful concepts in Gaussian processes: the
variational approach to sparse approximation and the spectral representation of
Gaussian processes. This gives rise to an approximation that inherits the
benefits of the variational approach but with the representational power and
computational scalability of spectral representations. The work hinges on a key
result that there exist spectral features related to a finite domain of the
Gaussian process which exhibit almost-independent covariances. We derive these
expressions for Matern kernels in one dimension, and generalize to more
dimensions using kernels with specific structures. Under the assumption of
additive Gaussian noise, our method requires only a single pass through the
dataset, making for very fast and accurate computation. We fit a model to 4
million training points in just a few minutes on a standard laptop. With
non-conjugate likelihoods, our MCMC scheme reduces the cost of computation from
O(NM2) (for a sparse Gaussian process) to O(NM) per iteration, where N is the
number of data and M is the number of features.

本篇论文提出了一种组合变分方法和光谱表示的高斯过程近似算法，通过研究高斯过程的光谱特征和协方差，进行了相关推导和分析，并将该算法应用于 Matern 核和高维数据的处理中，结果表明该算法在计算速度和精度方面都表现出色。