Graph Neural Networks (GNNs), known as spectral graph filters, find a wide
range of applications in web networks. To bypass eigendecomposition, polynomial
graph filters are proposed to approximate graph filters by leveraging various
polynomial bases for filter training. However, no existing studies have
explored the diverse polynomial graph filters from a unified perspective for
optimization.
In this paper, we first unify polynomial graph filters, as well as the
optimal filters of identical degrees into the Krylov subspace of the same
order, thus providing equivalent expressive power theoretically. Next, we
investigate the asymptotic convergence property of polynomials from the unified
Krylov subspace perspective, revealing their limited adaptability in graphs
with varying heterophily degrees. Inspired by those facts, we design a novel
adaptive Krylov subspace approach to optimize polynomial bases with provable
controllability over the graph spectrum so as to adapt various heterophily
graphs. Subsequently, we propose AdaptKry, an optimized polynomial graph filter
utilizing bases from the adaptive Krylov subspaces. Meanwhile, in light of the
diverse spectral properties of complex graphs, we extend AdaptKry by leveraging
multiple adaptive Krylov bases without incurring extra training costs. As a
consequence, extended AdaptKry is able to capture the intricate characteristics
of graphs and provide insights into their inherent complexity. We conduct
extensive experiments across a series of real-world datasets. The experimental
results demonstrate the superior filtering capability of AdaptKry, as well as
the optimized efficacy of the adaptive Krylov basis.

该论文提出了一种新的自适应 Krylov 子空间方法，通过优化多项式基，在图谱上实现了对不同异质度的图进行自适应滤波，并利用扩展的 AdaptKry 捕捉图的复杂特征并提供对其内在复杂性的见解。

优化多项式图滤波器：一种新的自适应克莱洛夫子空间方法

Optimizing Polynomial Graph Filters: A Novel Adaptive Krylov Subspace  Approach

Partial least squares (PLS) is a dimensionality reduction technique
introduced in the field of chemometrics and successfully employed in many other
areas. The PLS components are obtained by maximizing the covariance between
linear combinations of the regressors and of the target variables. In this
work, we focus on its application to scalar regression problems. PLS regression
consists in finding the least squares predictor that is a linear combination of
a subset of the PLS components. Alternatively, PLS regression can be formulated
as a least squares problem restricted to a Krylov subspace. This equivalent
formulation is employed to analyze the distance between
${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$, the PLS
estimator of the vector of coefficients of the linear regression model based on
$L$ PLS components, and $\hat{\boldsymbol \beta}_{\mathrm{OLS}}$, the one
obtained by ordinary least squares (OLS), as a function of $L$. Specifically,
${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$ is the
vector of coefficients in the aforementioned Krylov subspace that is closest to
$\hat{\boldsymbol \beta}_{\mathrm{OLS}}$ in terms of the Mahalanobis distance
with respect to the covariance matrix of the OLS estimate. We provide a bound
on this distance that depends only on the distribution of the eigenvalues of
the regressor covariance matrix. Numerical examples on synthetic and real-world
data are used to illustrate how the distance between
${\hat{\boldsymbol\beta}\;}_{\mathrm{PLS}}^{\scriptscriptstyle {(L)}}$ and
$\hat{\boldsymbol \beta}_{\mathrm{OLS}}$ depends on the number of clusters in
which the eigenvalues of the regressor covariance matrix are grouped.

这篇研究论文主要讨论了偏最小二乘（PLS）方法及其在标量回归问题中的应用，分析了由 PLS 得到的回归系数向量与普通最小二乘（OLS）方法得到的回归系数向量之间的距离，提供了该距离的上界并展示了其与回归因子协方差矩阵特征值分组数之间的关系。

关于回归斯克有主成分分析和普通最小二乘法的关系的研究

Relation between PLS and OLS regression in terms of the eigenvalue  distribution of the regressor covariance matrix

In this paper, we propose a second order optimization method to learn models
where both the dimensionality of the parameter space and the number of training
samples is high. In our method, we construct on each iteration a Krylov
subspace formed by the gradient and an approximation to the Hessian matrix, and
then use a subset of the training data samples to optimize over this subspace.
As with the Hessian Free (HF) method of [7], the Hessian matrix is never
explicitly constructed, and is computed using a subset of data. In practice, as
in HF, we typically use a positive definite substitute for the Hessian matrix
such as the Gauss-Newton matrix. We investigate the effectiveness of our
proposed method on deep neural networks, and compare its performance to widely
used methods such as stochastic gradient descent, conjugate gradient descent
and L-BFGS, and also to HF. Our method leads to faster convergence than either
L-BFGS or HF, and generally performs better than either of them in
cross-validation accuracy. It is also simpler and more general than HF, as it
does not require a positive semi-definite approximation of the Hessian matrix
to work well nor the setting of a damping parameter. The chief drawback versus
HF is the need for memory to store a basis for the Krylov subspace.

本研究提出了一个针对高维模型和大量训练样本的二阶优化方法，使用 Krylov 子空间进行训练加速，并在深度神经网络中的效果优于 SGD、共轭梯度下降和 L-BFGS 等算法，且优于 Hessian Free 方法。