Human brains are commonly modeled as networks of Regions of Interest (ROIs)
and their connections for the understanding of brain functions and mental
disorders. Recently, Transformer-based models have been studied over different
types of data, including graphs, shown to bring performance gains widely. In
this work, we study Transformer-based models for brain network analysis. Driven
by the unique properties of data, we model brain networks as graphs with nodes
of fixed size and order, which allows us to (1) use connection profiles as node
features to provide natural and low-cost positional information and (2) learn
pair-wise connection strengths among ROIs with efficient attention weights
across individuals that are predictive towards downstream analysis tasks.
Moreover, we propose an Orthonormal Clustering Readout operation based on
self-supervised soft clustering and orthonormal projection. This design
accounts for the underlying functional modules that determine similar behaviors
among groups of ROIs, leading to distinguishable cluster-aware node embeddings
and informative graph embeddings. Finally, we re-standardize the evaluation
pipeline on the only one publicly available large-scale brain network dataset
of ABIDE, to enable meaningful comparison of different models. Experiment
results show clear improvements of our proposed Brain Network Transformer on
both the public ABIDE and our restricted ABCD datasets. The implementation is
available at this https URL

本文探讨了基于 Transformer 模型的脑网络分析方法。通过将脑网络构建为固定大小和顺序的图形，使用连接特征提供自然低成本的位置信息，学习不同个体之间具有高预测性的特征强度。基于自监督软聚类和正交投影的正交聚类读取操作包含确定相似行为的基础功能模块，导致具有区分性的簇感知节点嵌入和信息丰富的图嵌入。该方法在公开 ABIDE 和经过限制的 ABCD 数据集上实现了明显的改善，代码实现可以在该 https URL 获取。