The drastic performance degradation of Graph Neural Networks (GNNs) as the depth of the graph propagation layers exceeds 8-10 is widely attributed to a phenomenon of Over-smoothing. Although recent research suggests that Over-smoothing may not be the dominant reason for such a performance degradation, they have not provided rigorous analysis from a theoretical view, which warrants further investigation. In this paper, we systematically analyze the real dominant problem in deep GNNs and identify the issues that these GNNs towards addressing Over-smoothing essentially work on via empirical experiments and theoretical gradient analysis. We theoretically prove that the difficult training problem of deep MLPs is actually the main challenge, and various existing methods that supposedly tackle Over-smoothing actually improve the trainability of MLPs, which is the main reason for their performance gains. Our further investigation into trainability issues reveals that properly constrained smaller upper bounds of gradient flow notably enhance the trainability of GNNs. Experimental results on diverse datasets demonstrate consistency between our theoretical findings and empirical evidence. Our analysis provides new insights in constructing deep graph models.

本研究旨在解决深度图神经网络（GNN）在层数超过8-10时性能急剧下降的问题，强调常见的过平滑现象并非主要原因。通过理论分析和实验，本论文揭示了深度多层感知器的可训练性问题是主要挑战，并提出适当限制梯度流的上界可以显著提升GNN的可训练性。实验结果验证了我们的理论发现与实证证据的一致性，推动了深度图模型的构建新思路。

超越过平滑：揭示深度图神经网络的可训练性挑战