Graph neural networks (GNN) suffer from severe inefficiency. It is mainly
caused by the exponential growth of node dependency with the increase of
layers. It extremely limits the application of stochastic optimization
algorithms so that the training of GNN is usually time-consuming. To address
this problem, we propose to decouple a multi-layer GNN as multiple simple
modules for more efficient training, which is comprised of classical forward
training (FT)and designed backward training (BT). Under the proposed framework,
each module can be trained efficiently in FT by stochastic algorithms without
distortion of graph information owing to its simplicity. To avoid the only
unidirectional information delivery of FT and sufficiently train shallow
modules with the deeper ones, we develop a backward training mechanism that
makes the former modules perceive the latter modules. The backward training
introduces the reversed information delivery into the decoupled modules as well
as the forward information delivery. To investigate how the decoupling and
greedy training affect the representational capacity, we theoretically prove
that the error produced by linear modules will not accumulate on unsupervised
tasks in most cases. The theoretical and experimental results show that the
proposed framework is highly efficient with reasonable performance.

提出了一种将多层 GNN 分解为多个简单模块的框架，并采用前向和后向训练机制进行高效训练，可以避免由于层数增加导致的依赖性快速增长和大量时间消耗。通过理论证明和实验结果表明，该框架的性能可行且高效。