Cancer survival prediction is a challenging task that involves analyzing of
the tumor microenvironment within Whole Slide Image (WSI). Previous methods
cannot effectively capture the intricate interaction features among instances
within the local area of WSI. Moreover, existing methods for cancer survival
prediction based on WSI often fail to provide better clinically meaningful
predictions. To overcome these challenges, we propose a Sparse Context-aware
Multiple Instance Learning (SCMIL) framework for predicting cancer survival
probability distributions. SCMIL innovatively segments patches into various
clusters based on their morphological features and spatial location
information, subsequently leveraging sparse self-attention to discern the
relationships between these patches with a context-aware perspective.
Considering many patches are irrelevant to the task, we introduce a learnable
patch filtering module called SoftFilter, which ensures that only interactions
between task-relevant patches are considered. To enhance the clinical relevance
of our prediction, we propose a register-based mixture density network to
forecast the survival probability distribution for individual patients. We
evaluate SCMIL on two public WSI datasets from the The Cancer Genome Atlas
(TCGA) specifically focusing on lung adenocarcinom (LUAD) and kidney renal
clear cell carcinoma (KIRC). Our experimental results indicate that SCMIL
outperforms current state-of-the-art methods for survival prediction, offering
more clinically meaningful and interpretable outcomes. Our code is accessible
at this https URL

通过稀疏上下文感知多实例学习框架，我们预测癌症存活概率分布，创新地将 WSI 的补丁根据形态特征和空间位置信息进行聚类，借助稀疏自注意力确定这些补丁之间的关系，使用可学习的补丁过滤模块 SoftFilter 仅考虑与任务相关的补丁之间的相互作用，并利用基于注册的混合密度网络预测个体患者的存活概率分布，实验证明我们的方法在存活预测方面胜过当前最先进的方法，提供更具临床意义和可解释性的结果。

SCMIL: 稀疏上下文感知多实例学习用于预测全切片图像中的癌症生存概率分布

SCMIL: Sparse Context-aware Multiple Instance Learning for Predicting  Cancer Survival Probability Distribution in Whole Slide Images

Multimodal learning significantly benefits cancer survival prediction,
especially the integration of pathological images and genomic data. Despite
advantages of multimodal learning for cancer survival prediction, massive
redundancy in multimodal data prevents it from extracting discriminative and
compact information: (1) An extensive amount of intra-modal task-unrelated
information blurs discriminability, especially for gigapixel whole slide images
(WSIs) with many patches in pathology and thousands of pathways in genomic
data, leading to an ``intra-modal redundancy" issue. (2) Duplicated information
among modalities dominates the representation of multimodal data, which makes
modality-specific information prone to being ignored, resulting in an
``inter-modal redundancy" issue. To address these, we propose a new framework,
Prototypical Information Bottlenecking and Disentangling (PIBD), consisting of
Prototypical Information Bottleneck (PIB) module for intra-modal redundancy and
Prototypical Information Disentanglement (PID) module for inter-modal
redundancy. Specifically, a variant of information bottleneck, PIB, is proposed
to model prototypes approximating a bunch of instances for different risk
levels, which can be used for selection of discriminative instances within
modality. PID module decouples entangled multimodal data into compact distinct
components: modality-common and modality-specific knowledge, under the guidance
of the joint prototypical distribution. Extensive experiments on five cancer
benchmark datasets demonstrated our superiority over other methods.

多模态学习对癌症生存预测有重要影响，尤其是病理图像和基因组数据的整合。本文提出了一种新的框架，用于解决多模态数据中的冗余性问题，并在五个癌症基准数据集上进行了广泛实验，证明其在其他方法上的优越性。