Graph deep learning (GDL) has demonstrated impressive performance in
predicting population-based brain disorders (BDs) through the integration of
both imaging and non-imaging data. However, the effectiveness of GDL based
methods heavily depends on the quality of modeling the multi-modal population
graphs and tends to degrade as the graph scale increases. Furthermore, these
methods often constrain interactions between imaging and non-imaging data to
node-edge interactions within the graph, overlooking complex inter-modal
correlations, leading to suboptimal outcomes. To overcome these challenges, we
propose MM-GTUNets, an end-to-end graph transformer based multi-modal graph
deep learning (MMGDL) framework designed for brain disorders prediction at
large scale. Specifically, to effectively leverage rich multi-modal information
related to diseases, we introduce Modality Reward Representation Learning
(MRRL) which adaptively constructs population graphs using a reward system.
Additionally, we employ variational autoencoder to reconstruct latent
representations of non-imaging features aligned with imaging features. Based on
this, we propose Adaptive Cross-Modal Graph Learning (ACMGL), which captures
critical modality-specific and modality-shared features through a unified
GTUNet encoder taking advantages of Graph UNet and Graph Transformer, and
feature fusion module. We validated our method on two public multi-modal
datasets ABIDE and ADHD-200, demonstrating its superior performance in
diagnosing BDs. Our code is available at this https URL

本研究提出了 MM-GTUNets，一种基于图变换器的多模态图深度学习（MMGDL）框架，用于大规模脑疾病预测，并通过验证在 ABIDE 和 ADHD-200 两个公共多模态数据集上证明了其优越性能。