Our study reveals new theoretical insights into over-smoothing and feature
over-correlation in deep graph neural networks. We show the prevalence of
invariant subspaces, demonstrating a fixed relative behavior that is unaffected
by feature transformations. Our work clarifies recent observations related to
convergence to a constant state and a potential over-separation of node states,
as the amplification of subspaces only depends on the spectrum of the
aggregation function. In linear scenarios, this leads to node representations
being dominated by a low-dimensional subspace with an asymptotic convergence
rate independent of the feature transformations. This causes a rank collapse of
the node representations, resulting in over-smoothing when smooth vectors span
this subspace, and over-correlation even when over-smoothing is avoided. Guided
by our theory, we propose a sum of Kronecker products as a beneficial property
that can provably prevent over-smoothing, over-correlation, and rank collapse.
We empirically extend our insights to the non-linear case, demonstrating the
inability of existing models to capture linearly independent features.

我们的研究揭示了深度图神经网络中的超平滑和特征过相关性的新的理论洞见。我们展示了不变子空间的普遍性，证明了固定的相对行为不受特征变换的影响。在线性场景中，这导致节点表示被低维子空间主导，其渐近收敛速率与特征变换无关，这导致节点表示的秩崩溃，从而在平滑向量跨越该子空间时产生超平滑，在避免超平滑的情况下产生超相关。根据我们的理论，我们提出了作为有益属性的 Kronecker 乘积之和，可以明确防止超平滑、超相关和秩崩溃。我们从实证角度将我们的洞见扩展到非线性情况，证明了现有模型无法捕捉线性独立特征。