Neural tangent kernels (NTKs) provide a theoretical regime to analyze the
learning and generalization behavior of over-parametrized neural networks. For
a supervised learning task, the association between the eigenvectors of the NTK
kernel and given data (a concept referred to as alignment in this paper) can
govern the rate of convergence of gradient descent, as well as generalization
to unseen data. Building upon this concept, we investigate NTKs and alignment
in the context of graph neural networks (GNNs), where our analysis reveals that
optimizing alignment translates to optimizing the graph representation or the
graph shift operator in a GNN. Our results further establish the theoretical
guarantees on the optimality of the alignment for a two-layer GNN and these
guarantees are characterized by the graph shift operator being a function of
the cross-covariance between the input and the output data. The theoretical
insights drawn from the analysis of NTKs are validated by our experiments
focused on a multi-variate time series prediction task for a publicly available
dataset. Specifically, they demonstrate that GNNs with cross-covariance as the
graph shift operator indeed outperform those that operate on the covariance
matrix from only the input data.

神经切向核、对齐、图神经网络、图移位算子和交叉协方差是本研究的关键词，该论文的主要内容是分析过度参数化神经网络的学习和泛化行为的理论机制，以及在图神经网络中优化对齐对图表示和图移位算子的重要性，并通过实验证明使用交叉协方差作为图移位算子的图神经网络在多变量时间序列预测任务中优于仅使用输入数据协方差矩阵的网络。

神经切向核激发带有交叉协方差图的图神经网络

Neural Tangent Kernels Motivate Graph Neural Networks with  Cross-Covariance Graphs

In many domains data is currently represented as graphs and therefore, the
graph representation of this data becomes increasingly important in machine
learning. Network data is, implicitly or explicitly, always represented using a
graph shift operator (GSO) with the most common choices being the adjacency,
Laplacian matrices and their normalisations. In this paper, a novel
parametrised GSO (PGSO) is proposed, where specific parameter values result in
the most commonly used GSOs and message-passing operators in graph neural
network (GNN) frameworks. The PGSO is suggested as a replacement of the
standard GSOs that are used in state-of-the-art GNN architectures and the
optimisation of the PGSO parameters is seamlessly included in the model
training. It is proved that the PGSO has real eigenvalues and a set of real
eigenvectors independent of the parameter values and spectral bounds on the
PGSO are derived. PGSO parameters are shown to adapt to the sparsity of the
graph structure in a study on stochastic blockmodel networks, where they are
found to automatically replicate the GSO regularisation found in the
literature. On several real-world datasets the accuracy of state-of-the-art GNN
architectures is improved by the inclusion of the PGSO in both node- and
graph-classification tasks.

提出了一种新的参数化图移位算子（PGSO），可以有效地替代现有的 GNN 架构中使用的标准 GSO，并且 PGSO 参数的优化被无缝地纳入模型训练中，研究了在随机块模型网络中，PGSO 参数会自适应地复制文献中发现的 GSO 正则化。在多个真实世界的数据集上，使用 PGSO 在节点分类和图分类任务中均可以提高现有 GNN 架构的准确性。