Masked graph autoencoders have emerged as a powerful graph self-supervised
learning method that has yet to be fully explored. In this paper, we unveil
that the existing discrete edge masking and binary link reconstruction
strategies are insufficient to learn topologically informative representations,
from the perspective of message propagation on graph neural networks. These
limitations include blocking message flows, vulnerability to over-smoothness,
and suboptimal neighborhood discriminability. Inspired by these understandings,
we explore non-discrete edge masks, which are sampled from a continuous and
dispersive probability distribution instead of the discrete Bernoulli
distribution. These masks restrict the amount of output messages for each edge,
referred to as "bandwidths". We propose a novel, informative, and effective
topological masked graph autoencoder using bandwidth masking and a layer-wise
bandwidth prediction objective. We demonstrate its powerful graph topological
learning ability both theoretically and empirically. Our proposed framework
outperforms representative baselines in both self-supervised link prediction
(improving the discrete edge reconstructors by at most 20%) and node
classification on numerous datasets, solely with a structure-learning pretext.
Our implementation is available at this https URL

使用带宽掩蔽和逐层带宽预测目标，我们提出了一种新颖且有效的拓扑掩蔽图自编码器，理论上和经验上证明了其强大的图拓扑学习能力，通过自我监督的链接预测和节点分类，在各种数据集上优于代表性的基准模型。

具有非离散带宽的掩盖图自编码器

Masked Graph Autoencoder with Non-discrete Bandwidths

Graph self-supervised learning (SSL), including contrastive and generative
approaches, offers great potential to address the fundamental challenge of
label scarcity in real-world graph data. Among both sets of graph SSL
techniques, the masked graph autoencoders (e.g., GraphMAE)--one type of
generative method--have recently produced promising results. The idea behind
this is to reconstruct the node features (or structures)--that are randomly
masked from the input--with the autoencoder architecture. However, the
performance of masked feature reconstruction naturally relies on the
discriminability of the input features and is usually vulnerable to disturbance
in the features. In this paper, we present a masked self-supervised learning
framework GraphMAE2 with the goal of overcoming this issue. The idea is to
impose regularization on feature reconstruction for graph SSL. Specifically, we
design the strategies of multi-view random re-mask decoding and latent
representation prediction to regularize the feature reconstruction. The
multi-view random re-mask decoding is to introduce randomness into
reconstruction in the feature space, while the latent representation prediction
is to enforce the reconstruction in the embedding space. Extensive experiments
show that GraphMAE2 can consistently generate top results on various public
datasets, including at least 2.45% improvements over state-of-the-art baselines
on ogbn-Papers100M with 111M nodes and 1.6B edges.

本文提出了一种基于掩蔽自监督学习框架 GraphMAE2，利用多视角随机重新覆盖解码策略和潜在表示预测策略，并通过广泛的实验表明其在各种公共数据集上始终产生顶级结果。