Foundation Models (FMs) serve as a general class for the development of
artificial intelligence systems, offering broad potential for generalization
across a spectrum of downstream tasks. Despite extensive research into
self-supervised learning as the cornerstone of FMs, several outstanding issues
persist in Graph Foundation Models that rely on graph self-supervised learning,
namely: 1) Homogenization. The extent of generalization capability on
downstream tasks remains unclear. 2) Scalability. It is unknown how effectively
these models can scale to large datasets. 3) Efficiency. The training time and
memory usage of these models require evaluation. 4) Training Stop Criteria.
Determining the optimal stopping strategy for pre-training across multiple
tasks to maximize performance on downstream tasks. To address these questions,
we have constructed a rigorous benchmark that thoroughly analyzes and studies
the generalization and scalability of self-supervised Graph Neural Network
(GNN) models. Regarding generalization, we have implemented and compared the
performance of various self-supervised GNN models, trained to generate node
representations, across tasks such as node classification, link prediction, and
node clustering. For scalability, we have compared the performance of various
models after training using full-batch and mini-batch strategies. Additionally,
we have assessed the training efficiency of these models by conducting
experiments to test their GPU memory usage and throughput. Through these
experiments, we aim to provide insights to motivate future research. The code
for this benchmark is publicly available at this https URL

基于自主学习的图神经网络模型的广义化和可扩展性进行了全面分析和研究，通过生成节点表示，在节点分类、链接预测和节点聚类等任务上实现了各种自主学习图神经网络模型的性能比较，并比较了全批次和小批次训练策略后各个模型的性能，同时评估了这些模型的训练效率。

GraphFM：图基模型的全面基准测试

GraphFM: A Comprehensive Benchmark for Graph Foundation Model

Graph self-supervised learning is now a go-to method for pre-training graph
foundation models, including graph neural networks, graph transformers, and
more recent large language model (LLM)-based graph models. There is a wide
variety of knowledge patterns embedded in the structure and properties of
graphs which may be used for pre-training, but we lack a systematic overview of
self-supervised pre-training tasks from the perspective of graph knowledge. In
this paper, we comprehensively survey and analyze the pre-training tasks of
graph foundation models from a knowledge-based perspective, consisting of
microscopic (nodes, links, etc) and macroscopic knowledge (clusters, global
structure, etc). It covers a total of 9 knowledge categories and 25
pre-training tasks, as well as various downstream task adaptation strategies.
Furthermore, an extensive list of the related papers with detailed metadata is
provided at this https URL

这篇论文综合了基于图知识的微观和宏观视角，对图基础模型的自我监督预训练任务进行了综述和分析，并包括了 9 个知识类别和 25 个预训练任务，以及各种下游任务适应策略。

图基础模型自我监督预训练调研：基于知识的视角

A Survey on Self-Supervised Pre-Training of Graph Foundation Models: A  Knowledge-Based Perspective

Emerging as fundamental building blocks for diverse artificial intelligence
applications, foundation models have achieved notable success across natural
language processing and many other domains. Parallelly, graph machine learning
has witnessed a transformative shift, with shallow methods giving way to deep
learning approaches. The emergence and homogenization capabilities of
foundation models have piqued the interest of graph machine learning
researchers, sparking discussions about developing the next graph learning
paradigm that is pre-trained on broad graph data and can be adapted to a wide
range of downstream graph tasks. However, there is currently no clear
definition and systematic analysis for this type of work. In this article, we
propose the concept of graph foundation models (GFMs), and provide the first
comprehensive elucidation on their key characteristics and technologies.
Following that, we categorize existing works towards GFMs into three categories
based on their reliance on graph neural networks and large language models.
Beyond providing a comprehensive overview of the current landscape of graph
foundation models, this article also discusses potential research directions
for this evolving field.

提出了图基础模型（GFMs）的概念，并对其关键特征和技术进行首次全面阐述，然后将现有工作根据其依赖性分为三类，即基于图神经网络和大语言模型。本文不仅提供了对当前图基础模型领域的综合概述，还讨论了这一发展中领域的潜在研究方向。