Survival risk stratification is an important step in clinical decision making
for breast cancer management. We propose a novel deep learning approach for
this purpose by integrating histopathological imaging, genetic and clinical
data. It employs vision transformers, specifically the MaxViT model, for image
feature extraction, and self-attention to capture intricate image relationships
at the patient level. A dual cross-attention mechanism fuses these features
with genetic data, while clinical data is incorporated at the final layer to
enhance predictive accuracy. Experiments on the public TCGA-BRCA dataset show
that our model, trained using the negative log likelihood loss function, can
achieve superior performance with a mean C-index of 0.64, surpassing existing
methods. This advancement facilitates tailored treatment strategies,
potentially leading to improved patient outcomes.

通过将组织病理图像、遗传数据和临床数据整合，构建了一种新颖的深度学习方法，使用视觉转换器和自注意力机制在病人层面上提取图像特征和捕获图像关系，同时采用双交叉注意力机制结合遗传数据，并在最终层级上加入临床数据以提高预测准确性，通过在公共 TCGA-BRCA 数据集上的实验，表明使用负对数似然损失函数训练的模型能够取得卓越的性能，均值 C 指数为 0.64，超过现有方法，为制定个体化的治疗策略提供了便利，潜在地改善了患者预后。