Self-supervised learning with masked autoencoders has recently gained
popularity for its ability to produce effective image or textual
representations, which can be applied to various downstream tasks without
retraining. However, we observe that the current masked autoencoder models lack
good generalization ability on graph data. To tackle this issue, we propose a
novel graph masked autoencoder framework called GiGaMAE. Different from
existing masked autoencoders that learn node presentations by explicitly
reconstructing the original graph components (e.g., features or edges), in this
paper, we propose to collaboratively reconstruct informative and integrated
latent embeddings. By considering embeddings encompassing graph topology and
attribute information as reconstruction targets, our model could capture more
generalized and comprehensive knowledge. Furthermore, we introduce a mutual
information based reconstruction loss that enables the effective reconstruction
of multiple targets. This learning objective allows us to differentiate between
the exclusive knowledge learned from a single target and common knowledge
shared by multiple targets. We evaluate our method on three downstream tasks
with seven datasets as benchmarks. Extensive experiments demonstrate the
superiority of GiGaMAE against state-of-the-art baselines. We hope our results
will shed light on the design of foundation models on graph-structured data.
Our code is available at: this https URL

我们提出了一种名为 GiGaMAE 的新型图掩蔽自编码器框架，通过协同重构信息化和综合的潜在嵌入，从而解决了当前图数据上遇到的推理能力不强的问题。通过将图拓扑和属性信息包含在嵌入中，作为重构目标，我们的模型能够捕捉到更广义和综合的知识，并引入基于互信息的重构损失，使得能够有效重构多个目标，实验结果表明 GiGaMAE 在多个基准测试中表现优于现有基线模型。