It has been discovered that Graph Convolutional Networks (GCNs) encounter a
remarkable drop in performance when multiple layers are piled up. The main
factor that accounts for why deep GCNs fail lies in over-smoothing, which
isolates the network output from the input with the increase of network depth,
weakening expressivity and trainability. In this paper, we start by
investigating refined measures upon DropEdge -- an existing simple yet
effective technique to relieve over-smoothing. We term our method as DropEdge++
for its two structure-aware samplers in contrast to DropEdge: layer-dependent
sampler and feature-dependent sampler. Regarding the layer-dependent sampler,
we interestingly find that increasingly sampling edges from the bottom layer
yields superior performance than the decreasing counterpart as well as
DropEdge. We theoretically reveal this phenomenon with Mean-Edge-Number (MEN),
a metric closely related to over-smoothing. For the feature-dependent sampler,
we associate the edge sampling probability with the feature similarity of node
pairs, and prove that it further correlates the convergence subspace of the
output layer with the input features. Extensive experiments on several node
classification benchmarks, including both full- and semi- supervised tasks,
illustrate the efficacy of DropEdge++ and its compatibility with a variety of
backbones by achieving generally better performance over DropEdge and the
no-drop version.

通过实验研究，本文介绍并提出了一种名为 DropEdge++ 的新颖技术，在多层图卷积网络中有效缓解过度平滑的问题，进而提升节点分类的性能。

面向深度图卷积网络的结构感知 DropEdge

Structure-Aware DropEdge Towards Deep Graph Convolutional Networks

Increasing the depth of GCN, which is expected to permit more expressivity,
is shown to incur performance detriment especially on node classification. The
main cause of this lies in over-smoothing. The over-smoothing issue drives the
output of GCN towards a space that contains limited distinguished information
among nodes, leading to poor expressivity. Several works on refining the
architecture of deep GCN have been proposed, but it is still unknown in theory
whether or not these refinements are able to relieve over-smoothing. In this
paper, we first theoretically analyze how general GCNs act with the increase in
depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We find that
all these models are characterized by a universal process: all nodes converging
to a cuboid. Upon this theorem, we propose DropEdge to alleviate over-smoothing
by randomly removing a certain number of edges at each training epoch.
Theoretically, DropEdge either reduces the convergence speed of over-smoothing
or relieves the information loss caused by dimension collapse. Experimental
evaluations on simulated dataset have visualized the difference in
over-smoothing between different GCNs. Moreover, extensive experiments on
several real benchmarks support that DropEdge consistently improves the
performance on a variety of both shallow and deep GCNs.

本研究分析了 GCN、GCN with bias、ResGCN 和 APPNP 模型的节点特征收敛过程，并提出了 DropEdge 来缓解过平滑问题，其在模拟数据和多个真实基准测试上均表现出显著性能提升。

针对通用图卷积网络的光滑问题解决

Tackling Over-Smoothing for General Graph Convolutional Networks

\emph{Over-fitting} and \emph{over-smoothing} are two main obstacles of
developing deep Graph Convolutional Networks (GCNs) for node classification. In
particular, over-fitting weakens the generalization ability on small dataset,
while over-smoothing impedes model training by isolating output representations
from the input features with the increase in network depth. This paper proposes
DropEdge, a novel and flexible technique to alleviate both issues. At its core,
DropEdge randomly removes a certain number of edges from the input graph at
each training epoch, acting like a data augmenter and also a message passing
reducer. Furthermore, we theoretically demonstrate that DropEdge either reduces
the convergence speed of over-smoothing or relieves the information loss caused
by it. More importantly, our DropEdge is a general skill that can be equipped
with many other backbone models (e.g. GCN, ResGCN, GraphSAGE, and JKNet) for
enhanced performance. Extensive experiments on several benchmarks verify that
DropEdge consistently improves the performance on a variety of both shallow and
deep GCNs. The effect of DropEdge on preventing over-smoothing is empirically
visualized and validated as well. Codes are released
on~https://github.com/DropEdge/DropEdge.

该论文提出了 DropEdge 技术来解决深层图卷积网络在节点分类中的过拟合和过度平滑问题。DropEdge 技术通过随机在每个训练迭代中移除一定数量的边缘节点来进行数据扩充和信息传递减少，旨在提高广义化效果和模型收敛速度。该技术还可以与其他多个骨干模型一起使用，使得在各种浅层和深层图卷积网络中都能一致提高模型的性能表现。