Alzheimer's disease (AD) is the most prevalent neurodegenerative disease; yet
its currently available treatments are limited to stopping disease progression.
Moreover, effectiveness of these treatments is not guaranteed due to the
heterogenetiy of the disease. Therefore, it is essential to be able to identify
the disease subtypes at a very early stage. Current data driven approaches are
able to classify the subtypes at later stages of AD or related disorders, but
struggle when predicting at the asymptomatic or prodromal stage. Moreover, most
existing models either lack explainability behind the classification or only
use a single modality for the assessment, limiting scope of its analysis. Thus,
we propose a multimodal framework that uses early-stage indicators such as
imaging, genetics and clinical assessments to classify AD patients into
subtypes at early stages. Similarly, we build prompts and use large language
models, such as ChatGPT, to interpret the findings of our model. In our
framework, we propose a tri-modal co-attention mechanism (Tri-COAT) to
explicitly learn the cross-modal feature associations. Our proposed model
outperforms baseline models and provides insight into key cross-modal feature
associations supported by known biological mechanisms.

阿尔茨海默病（AD）是最常见的神经退行性疾病，但目前的可用治疗方法仅限于停止疾病进展。此外，由于该疾病的异质性，这些治疗方法的有效性并不保证。因此，能够在早期阶段确定疾病亚型至关重要。我们提出了一个多模态框架，使用早期指标如图像、遗传学和临床评估来对早期阶段的 AD 患者进行亚型分类。同时，我们建立提示并使用大型语言模型（如 ChatGPT）来解释我们模型的发现。在我们的框架中，我们提出了一个三模态共同关注机制（Tri-COAT），以明确学习跨模态特征关联。我们提出的模型优于基线模型，并提供了支持已知生物学机制的关键跨模态特征关联的洞察。