Graph Neural Networks (GNNs) have emerged as the most powerful weapon for
various graph tasks due to the message-passing mechanism's great local
information aggregation ability. However, over-smoothing has always hindered
GNNs from going deeper and capturing multi-hop neighbors. Unlike GNNs,
Transformers can model global information and multi-hop interactions via
multi-head self-attention and a proper Transformer structure can show more
immunity to the over-smoothing problem. So, can we propose a novel framework to
combine GNN and Transformer, integrating both GNN's local information
aggregation and Transformer's global information modeling ability to eliminate
the over-smoothing problem? To realize this, this paper proposes a
collaborative learning scheme for GNN-Transformer and constructs GTC
architecture. GTC leverages the GNN and Transformer branch to encode node
information from different views respectively, and establishes contrastive
learning tasks based on the encoded cross-view information to realize
self-supervised heterogeneous graph representation. For the Transformer branch,
we propose Metapath-aware Hop2Token and CG-Hetphormer, which can cooperate with
GNN to attentively encode neighborhood information from different levels. As
far as we know, this is the first attempt in the field of graph representation
learning to utilize both GNN and Transformer to collaboratively capture
different view information and conduct cross-view contrastive learning. The
experiments on real datasets show that GTC exhibits superior performance
compared with state-of-the-art methods. Codes can be available at
this https URL

提出了一种结合 GNN 和 Transformer 的新框架，利用 GNN 的局部信息汇聚和 Transformer 的全局信息建模能力解决过度平滑问题，构建了 GTC 体系结构，通过对跨视图信息进行自我监督的对比学习任务实现异构图形表示学习。