Hypergraphs are marked by complex topology, expressing higher-order
interactions among multiple nodes with hyperedges, and better capturing the
topology is essential for effective representation learning. Recent advances in
generative self-supervised learning (SSL) suggest that hypergraph neural
networks learned from generative self supervision have the potential to
effectively encode the complex hypergraph topology. Designing a generative SSL
strategy for hypergraphs, however, is not straightforward. Questions remain
with regard to its generative SSL task, connection to downstream tasks, and
empirical properties of learned representations. In light of the promises and
challenges, we propose a novel generative SSL strategy for hypergraphs. We
first formulate a generative SSL task on hypergraphs, hyperedge filling, and
highlight its theoretical connection to node classification. Based on the
generative SSL task, we propose a hypergraph SSL method, HypeBoy. HypeBoy
learns effective general-purpose hypergraph representations, outperforming 16
baseline methods across 11 benchmark datasets.

提出了一种针对超图的新型生成性自监督学习策略，通过超边填充的生成性自监督任务来学习复杂的超图拓扑，其中的方法 HypeBoy 在 11 个基准数据集上优于 16 种基线方法，学习了有效的通用超图表示。

HypeBoy: 超图上的生成式自我监督学习

HypeBoy: Generative Self-Supervised Representation Learning on  Hypergraphs

Generative self-supervised learning (SSL) has exhibited significant potential
and garnered increasing interest in graph learning. In this study, we aim to
explore the problem of generative SSL in the context of heterogeneous graph
learning (HGL). The previous SSL approaches for heterogeneous graphs have
primarily relied on contrastive learning, necessitating the design of complex
views to capture heterogeneity. However, existing generative SSL methods have
not fully leveraged the capabilities of generative models to address the
challenges of HGL. In this paper, we present HGCVAE, a novel contrastive
variational graph auto-encoder that liberates HGL from the burden of intricate
heterogeneity capturing. Instead of focusing on complicated heterogeneity,
HGCVAE harnesses the full potential of generative SSL. HGCVAE innovatively
consolidates contrastive learning with generative SSL, introducing several key
innovations. Firstly, we employ a progressive mechanism to generate
high-quality hard negative samples for contrastive learning, utilizing the
power of variational inference. Additionally, we present a dynamic mask
strategy to ensure effective and stable learning. Moreover, we propose an
enhanced scaled cosine error as the criterion for better attribute
reconstruction. As an initial step in combining generative and contrastive SSL,
HGCVAE achieves remarkable results compared to various state-of-the-art
baselines, confirming its superiority.

HGCVAE 是一种新颖的对比变分图自编码器，通过创新地结合了生成性自我监督学习和对比性自我监督学习的方法，在处理异构图学习的挑战时表现出显著的结果。

HGCVAE: 整合生成与对比学习的异构图学习

HGCVAE: Integrating Generative and Contrastive Learning for  Heterogeneous Graph Learning

Self-supervised learning (SSL) has been extensively explored in recent years.
Particularly, generative SSL has seen emerging success in natural language
processing and other AI fields, such as the wide adoption of BERT and GPT.
Despite this, contrastive learning-which heavily relies on structural data
augmentation and complicated training strategies-has been the dominant approach
in graph SSL, while the progress of generative SSL on graphs, especially graph
autoencoders (GAEs), has thus far not reached the potential as promised in
other fields. In this paper, we identify and examine the issues that negatively
impact the development of GAEs, including their reconstruction objective,
training robustness, and error metric. We present a masked graph autoencoder
GraphMAE that mitigates these issues for generative self-supervised graph
pretraining. Instead of reconstructing graph structures, we propose to focus on
feature reconstruction with both a masking strategy and scaled cosine error
that benefit the robust training of GraphMAE. We conduct extensive experiments
on 21 public datasets for three different graph learning tasks. The results
manifest that GraphMAE-a simple graph autoencoder with careful designs-can
consistently generate outperformance over both contrastive and generative
state-of-the-art baselines. This study provides an understanding of graph
autoencoders and demonstrates the potential of generative self-supervised
pre-training on graphs.

本研究提出了一种掩蔽图自编码器 (GraphMAE)，解决了基于结构数据增强和复杂的训练策略的对比学习的不足之处，侧重于特征重建，并表现出优异的性能。