Masked graph autoencoder (MGAE) has emerged as a promising self-supervised
graph pre-training (SGP) paradigm due to its simplicity and effectiveness.
However, existing efforts perform the mask-then-reconstruct operation in the
raw data space as is done in computer vision (CV) and natural language
processing (NLP) areas, while neglecting the important non-Euclidean property
of graph data. As a result, the highly unstable local connection structures
largely increase the uncertainty in inferring masked data and decrease the
reliability of the exploited self-supervision signals, leading to inferior
representations for downstream evaluations. To address this issue, we propose a
novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the
certainty in inferring masked data and the reliability of the self-supervision
mechanism by further masking and reconstructing node samples in the high-order
latent feature space. Through both theoretical and empirical analyses, we have
discovered that performing a joint mask-then-reconstruct strategy in both
latent feature and raw data spaces could yield improved stability and
performance. To this end, we elaborately design a masked latent feature
completion scheme, which predicts latent features of masked nodes under the
guidance of high-order sample correlations that are hard to be observed from
the raw data perspective. Specifically, we first adopt a latent feature
predictor to predict the masked latent features from the visible ones. Next, we
encode the raw data of masked samples with a momentum graph encoder and
subsequently employ the resulting representations to improve predicted results
through latent feature matching. Extensive experiments on seventeen datasets
have demonstrated the effectiveness and robustness of RARE against
state-of-the-art (SOTA) competitors across three downstream tasks.

本文提出了一种新的自监督图形预训练方法 RARE，它在高阶潜在特征空间中进一步掩蔽和重构节点样本以提高推断掩蔽数据的确定性和自监督机制的可靠性，并采用的联合掩蔽 - 重构策略可以在潜在特征和原始数据空间中产生改进的稳定性和性能，在 17 个数据集上的大量实验中验证了 RARE 的有效性和鲁棒性。