We introduce \textit{GraphGPT}, a novel model for Graph learning by
self-supervised Generative Pre-training Transformers. Our model transforms each
graph or sampled subgraph into a sequence of tokens representing the node, edge
and attributes reversibly using the Eulerian path first. Then we feed the
tokens into a standard transformer decoder and pre-train it with the
next-token-prediction (NTP) task. Lastly, we fine-tune the GraphGPT model with
the supervised tasks. This intuitive, yet effective model achieves superior or
close results to the state-of-the-art methods for the graph-, edge- and
node-level tasks on the large scale molecular dataset PCQM4Mv2, the
protein-protein association dataset ogbl-ppa and the ogbn-proteins dataset from
the Open Graph Benchmark (OGB). Furthermore, the generative pre-training
enables us to train GraphGPT up to 400M+ parameters with consistently
increasing performance, which is beyond the capability of GNNs and previous
graph transformers. The source code and pre-trained checkpoints will be
released soon\footnote{https://github.com/alibaba/graph-gpt} to pave the
way for the graph foundation model research, and also to assist the scientific
discovery in pharmaceutical, chemistry, material and bio-informatics domains,
etc.

我们介绍了一种名为 GraphGPT 的自我监督建模方法，该方法通过将图或抽样子图转化为表示节点、边和属性的可逆令牌序列，并使用欧拉路径优先进行预训练，最后通过监督学习任务进行微调，该模型在大规模分子数据集 PCQM4Mv2、蛋白质相互作用数据集 ogbl-ppa 以及开放图基准数据集 ogbn-proteins 上取得了接近或优于现有方法的结果，并且能够训练超过 400M 个参数的 GraphGPT，具有一致提升的性能，这超出了 GNN 和先前的图变换模型的能力。