Normalization is known to help the optimization of deep neural networks.
Curiously, different architectures require specialized normalization methods.
In this paper, we study what normalization is effective for Graph Neural
Networks (GNNs). First, we adapt and evaluate the existing methods from other
domains to GNNs. Faster convergence is achieved with InstanceNorm compared to
BatchNorm and LayerNorm. We provide an explanation by showing that InstanceNorm
serves as a preconditioner for GNNs, but such preconditioning effect is weaker
with BatchNorm due to the heavy batch noise in graph datasets. Second, we show
that the shift operation in InstanceNorm results in an expressiveness
degradation of GNNs for highly regular graphs. We address this issue by
proposing GraphNorm with a learnable shift. Empirically, GNNs with GraphNorm
converge faster compared to GNNs using other normalization. GraphNorm also
improves the generalization of GNNs, achieving better performance on graph
classification benchmarks.

本文研究了对图神经网络进行归一化的有效方法，将 InstanceNorm 与 BatchNorm 和 LayerNorm 进行比较，证明了 InstanceNorm 对 GNNs 的预处理效果最好。同时，我们也发现 InstanceNorm 的偏移操作可能会导致 GNNs 的表现能力下降，因此引入了可学习的 GraphNorm 对表现力下降的问题进行纠正，实验证明使用 GraphNorm 推出的 GNN 准确率得到了显著提升。