Joint-Embedding Predictive Architectures (JEPAs) have recently emerged as a
novel and powerful technique for self-supervised representation learning. They
aim to learn an energy-based model by predicting the latent representation of a
target signal $y$ from a context signal $x$. JEPAs bypass the need for data
augmentation and negative samples, which are typically required by contrastive
learning, while avoiding the overfitting issues associated with
generative-based pretraining. In this paper, we show that graph-level
representations can be effectively modeled using this paradigm and propose
Graph-JEPA, the first JEPA for the graph domain. In particular, we employ
masked modeling to learn embeddings for different subgraphs of the input graph.
To endow the representations with the implicit hierarchy that is often present
in graph-level concepts, we devise an alternative training objective that
consists of predicting the coordinates of the encoded subgraphs on the unit
hyperbola in the 2D plane. Extensive validation shows that Graph-JEPA can learn
representations that are expressive and competitive in both graph
classification and regression problems.

Graph-JEPA 是首个针对图领域提出的 Joint-Embedding Predictive Architectures (JEPAs) 模型，通过掩码建模学习不同子图的嵌入表示，并且采用预测编码子图在二维平面上单位双曲线上坐标的替代训练目标，验证结果表明 Graph-JEPA 能够学习具有表达力并在图分类和回归问题中具有竞争力的表示。

联合嵌入预测架构的图级表示学习

Graph-level Representation Learning with Joint-Embedding Predictive  Architectures

This paper studies learning the representations of whole graphs in both
unsupervised and semi-supervised scenarios. Graph-level representations are
critical in a variety of real-world applications such as predicting the
properties of molecules and community analysis in social networks. Traditional
graph kernel based methods are simple, yet effective for obtaining fixed-length
representations for graphs but they suffer from poor generalization due to
hand-crafted designs. There are also some recent methods based on language
models (e.g. graph2vec) but they tend to only consider certain substructures
(e.g. subtrees) as graph representatives. Inspired by recent progress of
unsupervised representation learning, in this paper we proposed a novel method
called InfoGraph for learning graph-level representations. We maximize the
mutual information between the graph-level representation and the
representations of substructures of different scales (e.g., nodes, edges,
triangles). By doing so, the graph-level representations encode aspects of the
data that are shared across different scales of substructures. Furthermore, we
further propose InfoGraph*, an extension of InfoGraph for semi-supervised
scenarios. InfoGraph* maximizes the mutual information between unsupervised
graph representations learned by InfoGraph and the representations learned by
existing supervised methods. As a result, the supervised encoder learns from
unlabeled data while preserving the latent semantic space favored by the
current supervised task. Experimental results on the tasks of graph
classification and molecular property prediction show that InfoGraph is
superior to state-of-the-art baselines and InfoGraph* can achieve performance
competitive with state-of-the-art semi-supervised models.

本文研究无监督与半监督情境下，学习整个图的表示。通过最大化不同规模子结构（例如节点，边和三角形）的表示和图表示之间的互信息，提出了名为 InfoGraph 的图级表示学习方法。InfoGraph 优于现有的竞争基线。同时，我们还提出了一个名为 InfoGraph * 的扩展，主要用于半监督学习。

InfoGraph：利用互信息最大化进行无监督和半监督图级表示学习

InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation  Learning via Mutual Information Maximization

Recently a variety of methods have been developed to encode graphs into
low-dimensional vectors that can be easily exploited by machine learning
algorithms. The majority of these methods start by embedding the graph nodes
into a low-dimensional vector space, followed by using some scheme to aggregate
the node embeddings. In this work, we develop a new approach to learn
graph-level representations, which includes a combination of unsupervised and
supervised learning components. We start by learning a set of node
representations in an unsupervised fashion. Graph nodes are mapped into node
sequences sampled from random walk approaches approximated by the
Gumbel-Softmax distribution. Recurrent neural network (RNN) units are modified
to accommodate both the node representations as well as their neighborhood
information. Experiments on standard graph classification benchmarks
demonstrate that our proposed approach achieves superior or comparable
performance relative to the state-of-the-art algorithms in terms of convergence
speed and classification accuracy. We further illustrate the effectiveness of
the different components used by our approach.

本文提出了一种结合无监督和有监督学习组成的方法，通过利用 random walk 方法和 Gumbel-Softmax 分布将图节点映射到节点序列，然后使用修改后的 RNN 神经网络单位学习节点表示方法和它们的邻域信息，实验表明该方法优于或与现有算法相当，具有收敛速度快和准确度高的优点。