Masked graph autoencoder (MGAE) has emerged as a promising self-supervised
graph pre-training (SGP) paradigm due to its simplicity and effectiveness.
However, existing efforts perform the mask-then-reconstruct operation in the
raw data space as is done in computer vision (CV) and natural language
processing (NLP) areas, while neglecting the important non-Euclidean property
of graph data. As a result, the highly unstable local connection structures
largely increase the uncertainty in inferring masked data and decrease the
reliability of the exploited self-supervision signals, leading to inferior
representations for downstream evaluations. To address this issue, we propose a
novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the
certainty in inferring masked data and the reliability of the self-supervision
mechanism by further masking and reconstructing node samples in the high-order
latent feature space. Through both theoretical and empirical analyses, we have
discovered that performing a joint mask-then-reconstruct strategy in both
latent feature and raw data spaces could yield improved stability and
performance. To this end, we elaborately design a masked latent feature
completion scheme, which predicts latent features of masked nodes under the
guidance of high-order sample correlations that are hard to be observed from
the raw data perspective. Specifically, we first adopt a latent feature
predictor to predict the masked latent features from the visible ones. Next, we
encode the raw data of masked samples with a momentum graph encoder and
subsequently employ the resulting representations to improve predicted results
through latent feature matching. Extensive experiments on seventeen datasets
have demonstrated the effectiveness and robustness of RARE against
state-of-the-art (SOTA) competitors across three downstream tasks.

本文提出了一种新的自监督图形预训练方法 RARE，它在高阶潜在特征空间中进一步掩蔽和重构节点样本以提高推断掩蔽数据的确定性和自监督机制的可靠性，并采用的联合掩蔽 - 重构策略可以在潜在特征和原始数据空间中产生改进的稳定性和性能，在 17 个数据集上的大量实验中验证了 RARE 的有效性和鲁棒性。

稳健掩码图自编码器

RARE: Robust Masked Graph Autoencoder

Self-supervised learning (SSL) has been extensively explored in recent years.
Particularly, generative SSL has seen emerging success in natural language
processing and other AI fields, such as the wide adoption of BERT and GPT.
Despite this, contrastive learning-which heavily relies on structural data
augmentation and complicated training strategies-has been the dominant approach
in graph SSL, while the progress of generative SSL on graphs, especially graph
autoencoders (GAEs), has thus far not reached the potential as promised in
other fields. In this paper, we identify and examine the issues that negatively
impact the development of GAEs, including their reconstruction objective,
training robustness, and error metric. We present a masked graph autoencoder
GraphMAE that mitigates these issues for generative self-supervised graph
pretraining. Instead of reconstructing graph structures, we propose to focus on
feature reconstruction with both a masking strategy and scaled cosine error
that benefit the robust training of GraphMAE. We conduct extensive experiments
on 21 public datasets for three different graph learning tasks. The results
manifest that GraphMAE-a simple graph autoencoder with careful designs-can
consistently generate outperformance over both contrastive and generative
state-of-the-art baselines. This study provides an understanding of graph
autoencoders and demonstrates the potential of generative self-supervised
pre-training on graphs.

本研究提出了一种掩蔽图自编码器 (GraphMAE)，解决了基于结构数据增强和复杂的训练策略的对比学习的不足之处，侧重于特征重建，并表现出优异的性能。

自监督掩码图自编码器

GraphMAE: Self-Supervised Masked Graph Autoencoders

We introduce a novel masked graph autoencoder (MGAE) framework to perform
effective learning on graph structure data. Taking insights from
self-supervised learning, we randomly mask a large proportion of edges and try
to reconstruct these missing edges during training. MGAE has two core designs.
First, we find that masking a high ratio of the input graph structure, e.g.,
$70\%$, yields a nontrivial and meaningful self-supervisory task that benefits
downstream applications. Second, we employ a graph neural network (GNN) as an
encoder to perform message propagation on the partially-masked graph. To
reconstruct the large number of masked edges, a tailored cross-correlation
decoder is proposed. It could capture the cross-correlation between the head
and tail nodes of anchor edge in multi-granularity. Coupling these two designs
enables MGAE to be trained efficiently and effectively. Extensive experiments
on multiple open datasets (Planetoid and OGB benchmarks) demonstrate that MGAE
generally performs better than state-of-the-art unsupervised learning
competitors on link prediction and node classification.

本文提出了一种新颖的 MGAE 框架，通过对图结构进行随机遮掩边来重构这些缺失的边并利用 GNN 对部分遮掩的图进行传播来训练 MGAE 以达到有效学习图结构数据的目的，并在多个开放数据集上进行了广泛实验，表明 MGAE 在链接预测和节点分类方面总体性能优于最先进的无监督学习竞争对手。