Gaussian process latent variable models (GPLVMs) are a versatile family of
unsupervised learning models, commonly used for dimensionality reduction.
However, common challenges in modeling data with GPLVMs include inadequate
kernel flexibility and improper selection of the projection noise, which leads
to a type of model collapse characterized primarily by vague latent
representations that do not reflect the underlying structure of the data. This
paper addresses these issues by, first, theoretically examining the impact of
the projection variance on model collapse through the lens of a linear GPLVM.
Second, we address the problem of model collapse due to inadequate kernel
flexibility by integrating the spectral mixture (SM) kernel and a
differentiable random Fourier feature (RFF) kernel approximation, which ensures
computational scalability and efficiency through off-the-shelf automatic
differentiation tools for learning the kernel hyperparameters, projection
variance, and latent representations within the variational inference
framework. The proposed GPLVM, named advisedRFLVM, is evaluated across diverse
datasets and consistently outperforms various salient competing models,
including state-of-the-art variational autoencoders (VAEs) and GPLVM variants,
in terms of informative latent representations and missing data imputation.

本文通过理论分析证明了投影方差对模型崩溃的影响，并通过集成频谱混合核和可微分的随机傅里叶特征核逼近解决了内核灵活性不足导致的模型崩溃问题。提出的 GPLVM 模型在各种数据集上都取得了优异的表现，在信息丰富的潜在表示和缺失数据插补方面超越了各种竞争模型，包括最先进的变分自编码器和 GPLVM 变体。

高斯过程潜变量模型中的模型崩溃预防

Preventing Model Collapse in Gaussian Process Latent Variable Models

Machine learning based solvers have garnered much attention in physical
simulation and scientific computing, with a prominent example, physics-informed
neural networks (PINNs). However, PINNs often struggle to solve high-frequency
and multi-scale PDEs, which can be due to spectral bias during neural network
training. To address this problem, we resort to the Gaussian process (GP)
framework. To flexibly capture the dominant frequencies, we model the power
spectrum of the PDE solution with a student t mixture or Gaussian mixture. We
then apply the inverse Fourier transform to obtain the covariance function
(according to the Wiener-Khinchin theorem). The covariance derived from the
Gaussian mixture spectrum corresponds to the known spectral mixture kernel. We
are the first to discover its rationale and effectiveness for PDE solving.
Next,we estimate the mixture weights in the log domain, which we show is
equivalent to placing a Jeffreys prior. It automatically induces sparsity,
prunes excessive frequencies, and adjusts the remaining toward the ground
truth. Third, to enable efficient and scalable computation on massive
collocation points, which are critical to capture high frequencies, we place
the collocation points on a grid, and multiply our covariance function at each
input dimension. We use the GP conditional mean to predict the solution and its
derivatives so as to fit the boundary condition and the equation itself. As a
result, we can derive a Kronecker product structure in the covariance matrix.
We use Kronecker product properties and multilinear algebra to greatly promote
computational efficiency and scalability, without any low-rank approximations.
We show the advantage of our method in systematic experiments.

基于机器学习的求解器在物理模拟和科学计算中引起了广泛关注，本文介绍了一种基于高斯过程的方法来解决高频和多尺度的偏微分方程，利用混合核函数来灵活地捕捉主频，并通过在网格上放置采样点来加速计算，最终建立了一个 Kronecker 乘积结构的协方差矩阵。

用高斯过程求解高频率和多尺度偏微分方程

Solving High Frequency and Multi-Scale PDEs with Gaussian Processes

Early approaches to multiple-output Gaussian processes (MOGPs) relied on
linear combinations of independent, latent, single-output Gaussian processes
(GPs). This resulted in cross-covariance functions with limited parametric
interpretation, thus conflicting with the ability of single-output GPs to
understand lengthscales, frequencies and magnitudes to name a few. On the
contrary, current approaches to MOGP are able to better interpret the
relationship between different channels by directly modelling the
cross-covariances as a spectral mixture kernel with a phase shift. We extend
this rationale and propose a parametric family of complex-valued cross-spectral
densities and then build on Cram\'er's Theorem (the multivariate version of
Bochner's Theorem) to provide a principled approach to design multivariate
covariance functions. The so-constructed kernels are able to model delays among
channels in addition to phase differences and are thus more expressive than
previous methods, while also providing full parametric interpretation of the
relationship across channels. The proposed method is first validated on
synthetic data and then compared to existing MOGP methods on two real-world
examples.

提出了基于复值交叉频谱密度的参数化多元协方差函数，实现对通道之间时间延迟和相位差异的建模，进而提高了模型的表达能力。该方法在合成数据和真实数据集上进行了验证和比较。