Heterogeneous Graph Neural Networks (HGNNs) are a class of powerful deep
learning methods widely used to learn representations of heterogeneous graphs.
Despite the fast development of HGNNs, they still face some challenges such as
over-smoothing, and non-robustness. Previous studies have shown that these
problems can be reduced by using gradient regularization methods. However, the
existing gradient regularization methods focus on either graph topology or node
features. There is no universal approach to integrate these features, which
severely affects the efficiency of regularization. In addition, the inclusion
of gradient regularization into HGNNs sometimes leads to some problems, such as
an unstable training process, increased complexity and insufficient coverage
regularized information. Furthermore, there is still short of a complete
theoretical analysis of the effects of gradient regularization on HGNNs. In
this paper, we propose a novel gradient regularization method called Grug,
which iteratively applies regularization to the gradients generated by both
propagated messages and the node features during the message-passing process.
Grug provides a unified framework integrating graph topology and node features,
based on which we conduct a detailed theoretical analysis of their
effectiveness. Specifically, the theoretical analyses elaborate the advantages
of Grug: 1) Decreasing sample variance during the training process (Stability);
2) Enhancing the generalization of the model (Universality); 3) Reducing the
complexity of the model (Simplicity); 4) Improving the integrity and diversity
of graph information utilization (Diversity). As a result, Grug has the
potential to surpass the theoretical upper bounds set by DropMessage (AAAI-23
Distinguished Papers). In addition, we evaluate Grug on five public real-world
datasets with two downstream tasks.

本研究提出了一种名为 Grug 的新型梯度正则化方法，它可以在消息传递过程中对传播消息和节点特征生成的梯度都进行迭代正则化。Grug 提供了一个统一框架，可以集成图拓扑和节点特征，并基于此进行详细的理论分析它们的有效性。