Survival prediction is crucial for cancer patients as it provides early
prognostic information for treatment planning. Recently, deep survival models
based on deep learning and medical images have shown promising performance for
survival prediction. However, existing deep survival models are not well
developed in utilizing multi-modality images (e.g., PET-CT) and in extracting
region-specific information (e.g., the prognostic information in Primary Tumor
(PT) and Metastatic Lymph Node (MLN) regions). In view of this, we propose a
merging-diverging learning framework for survival prediction from
multi-modality images. This framework has a merging encoder to fuse
multi-modality information and a diverging decoder to extract region-specific
information. In the merging encoder, we propose a Hybrid Parallel
Cross-Attention (HPCA) block to effectively fuse multi-modality features via
parallel convolutional layers and cross-attention transformers. In the
diverging decoder, we propose a Region-specific Attention Gate (RAG) block to
screen out the features related to lesion regions. Our framework is
demonstrated on survival prediction from PET-CT images in Head and Neck (H&N)
cancer, by designing an X-shape merging-diverging hybrid transformer network
(named XSurv). Our XSurv combines the complementary information in PET and CT
images and extracts the region-specific prognostic information in PT and MLN
regions. Extensive experiments on the public dataset of HEad and neCK TumOR
segmentation and outcome prediction challenge (HECKTOR 2022) demonstrate that
our XSurv outperforms state-of-the-art survival prediction methods.

我们提出了一种融合 - 分叉学习框架，用于从多模态图像进行生存预测，并在融合编码器中使用混合并行交叉注意力块（HPCA）来融合多模态特征，并在分叉解码器中使用区域特定注意力门（RAG）块来筛选与病变区域相关的特征。我们的框架在头颈部 PET-CT 图像的生存预测中得到了验证，并在公共数据集上的实验结果表明，我们的网络（XSurv）优于现有的生存预测方法。