Graph Masked Autoencoders (GMAEs) have emerged as a notable self-supervised
learning approach for graph-structured data. Existing GMAE models primarily
focus on reconstructing node-level information, categorizing them as
single-scale GMAEs. This methodology, while effective in certain contexts,
tends to overlook the complex hierarchical structures inherent in many
real-world graphs. For instance, molecular graphs exhibit a clear hierarchical
organization in the form of the atoms-functional groups-molecules structure.
Hence, the inability of single-scale GMAE models to incorporate these
hierarchical relationships often leads to their inadequate capture of crucial
high-level graph information, resulting in a noticeable decline in performance.
To address this limitation, we propose Hierarchical Graph Masked AutoEncoders
(Hi-GMAE), a novel multi-scale GMAE framework designed to handle the
hierarchical structures within graphs. First, Hi-GMAE constructs a multi-scale
graph hierarchy through graph pooling, enabling the exploration of graph
structures across different granularity levels. To ensure masking uniformity of
subgraphs across these scales, we propose a novel coarse-to-fine strategy that
initiates masking at the coarsest scale and progressively back-projects the
mask to the finer scales. Furthermore, we integrate a gradual recovery strategy
with the masking process to mitigate the learning challenges posed by
completely masked subgraphs. Diverging from the standard graph neural network
(GNN) used in GMAE models, Hi-GMAE modifies its encoder and decoder into
hierarchical structures. This entails using GNN at the finer scales for
detailed local graph analysis and employing a graph transformer at coarser
scales to capture global information. Our experiments on 15 graph datasets
consistently demonstrate that Hi-GMAE outperforms 17 state-of-the-art
self-supervised competitors.

Hierarchical Graph Masked AutoEncoders (Hi-GMAE) is a multi-scale framework that addresses the limitations of single-scale Graph Masked Autoencoders (GMAEs) in incorporating hierarchical structures within graphs, resulting in improved performance compared to other self-supervised competitors.

Hi-GMAE：层次图掩码自编码器

Hi-GMAE: Hierarchical Graph Masked Autoencoders

Graph masked autoencoders (GMAE) have emerged as a significant advancement in
self-supervised pre-training for graph-structured data. Previous GMAE models
primarily utilize a straightforward random masking strategy for nodes or edges
during training. However, this strategy fails to consider the varying
significance of different nodes within the graph structure. In this paper, we
investigate the potential of leveraging the graph's structural composition as a
fundamental and unique prior in the masked pre-training process. To this end,
we introduce a novel structure-guided masking strategy (i.e., StructMAE),
designed to refine the existing GMAE models. StructMAE involves two steps: 1)
Structure-based Scoring: Each node is evaluated and assigned a score reflecting
its structural significance. Two distinct types of scoring manners are
proposed: predefined and learnable scoring. 2) Structure-guided Masking: With
the obtained assessment scores, we develop an easy-to-hard masking strategy
that gradually increases the structural awareness of the self-supervised
reconstruction task. Specifically, the strategy begins with random masking and
progresses to masking structure-informative nodes based on the assessment
scores. This design gradually and effectively guides the model in learning
graph structural information. Furthermore, extensive experiments consistently
demonstrate that our StructMAE method outperforms existing state-of-the-art
GMAE models in both unsupervised and transfer learning tasks. Codes are
available at this https URL

研究介绍了结构导向的掩膜策略（StructMAE），该策略通过评估节点的结构重要性并进行渐进式掩膜，从而有效地引导模型学习图结构信息，并在无监督学习和迁移学习任务中优于现有的 GMAE 模型。

图结构引导的掩模自编码器中的掩模位置选择

Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders

Generative self-supervised learning on graphs, particularly graph masked
autoencoders, has emerged as a popular learning paradigm and demonstrated its
efficacy in handling non-Euclidean data. However, several remaining issues
limit the capability of existing methods: 1) the disregard of uneven node
significance in masking, 2) the underutilization of holistic graph information,
3) the ignorance of semantic knowledge in the representation space due to the
exclusive use of reconstruction loss in the output space, and 4) the unstable
reconstructions caused by the large volume of masked contents. In light of
this, we propose UGMAE, a unified framework for graph masked autoencoders to
address these issues from the perspectives of adaptivity, integrity,
complementarity, and consistency. Specifically, we first develop an adaptive
feature mask generator to account for the unique significance of nodes and
sample informative masks (adaptivity). We then design a ranking-based structure
reconstruction objective joint with feature reconstruction to capture holistic
graph information and emphasize the topological proximity between neighbors
(integrity). After that, we present a bootstrapping-based similarity module to
encode the high-level semantic knowledge in the representation space,
complementary to the low-level reconstruction in the output space
(complementarity). Finally, we build a consistency assurance module to provide
reconstruction objectives with extra stabilized consistency targets
(consistency). Extensive experiments demonstrate that UGMAE outperforms both
contrastive and generative state-of-the-art baselines on several tasks across
multiple datasets.

通过采用 UGMAE 方法，在自学习图上进行图遮挡自编码器的统一框架，解决了现有方法中节点重要性不均匀、整体图信息利用不足、表示空间中的语义知识忽视以及遮挡内容过大导致的不稳定问题，实验证明 UGMAE 在多个数据集上的多项任务上优于对比和生成最先进基准模型。