We introduce a transformer-based GNN model, named UGformer, to learn graph
representations. In particular, we present two UGformer variants, wherein the
first variant (publicized in September 2019) is to leverage the transformer on
a set of sampled neighbors for each input node, while the second (publicized in
May 2021) is to leverage the transformer on all input nodes. Experimental
results demonstrate that the first UGformer variant achieves state-of-the-art
accuracies on benchmark datasets for graph classification in both inductive
setting and unsupervised transductive setting; and the second UGformer variant
obtains state-of-the-art accuracies for inductive text classification. The code
is available at: https://github.com/daiquocnguyen/Graph-Transformer.

本文介绍了一种基于 Transformer 的 GNN 模型，叫做 UGformer，用于学习图表示。UGformer 有两个变体，第一个变体利用 Transformer 对每个输入节点的一组采样邻居进行编码，第二个变体利用 Transformer 对所有输入节点进行编码。实验结果表明，第一个变体在归纳和非监督转导设置下实现了基准数据集上的最新最准确统计，第二个变体在归纳文本分类中获得了最高准确率。