Integrating the different data modalities of cancer patients can
significantly improve the predictive performance of patient survival. However,
most existing methods ignore the simultaneous utilization of rich semantic
features at different scales in pathology images. When collecting multimodal
data and extracting features, there is a likelihood of encountering
intra-modality missing data, introducing noise into the multimodal data. To
address these challenges, this paper proposes a new end-to-end framework,
FORESEE, for robustly predicting patient survival by mining multimodal
information. Specifically, the cross-fusion transformer effectively utilizes
features at the cellular level, tissue level, and tumor heterogeneity level to
correlate prognosis through a cross-scale feature cross-fusion method. This
enhances the ability of pathological image feature representation. Secondly,
the hybrid attention encoder (HAE) uses the denoising contextual attention
module to obtain the contextual relationship features and local detail features
of the molecular data. HAE's channel attention module obtains global features
of molecular data. Furthermore, to address the issue of missing information
within modalities, we propose an asymmetrically masked triplet masked
autoencoder to reconstruct lost information within modalities. Extensive
experiments demonstrate the superiority of our method over state-of-the-art
methods on four benchmark datasets in both complete and missing settings.

一个新的端到端框架 FORESEE，通过挖掘多模态信息来稳健地预测患者生存情况，同时利用细胞水平、组织水平和肿瘤异质性水平的丰富语义特征，解决了多模态数据的缺失问题，并通过交叉尺度特征融合方法，提高了病理图像特征表征能力。

FORESEE: 癌症生存鲁棒预测的多模态和多视图表示学习

FORESEE: Multimodal and Multi-view Representation Learning for Robust  Prediction of Cancer Survival

The VASARI MRI feature set is a quantitative system designed to standardise
glioma imaging descriptions. Though effective, deriving VASARI is
time-consuming and seldom used in clinical practice. This is a problem that
machine learning could plausibly automate. Using glioma data from 1172
patients, we developed VASARI-auto, an automated labelling software applied to
both open-source lesion masks and our openly available tumour segmentation
model. In parallel, two consultant neuroradiologists independently quantified
VASARI features in a subsample of 100 glioblastoma cases. We quantified: 1)
agreement across neuroradiologists and VASARI-auto; 2) calibration of
performance equity; 3) an economic workforce analysis; and 4) fidelity in
predicting patient survival. Tumour segmentation was compatible with the
current state of the art and equally performant regardless of age or sex. A
modest inter-rater variability between in-house neuroradiologists was
comparable to between neuroradiologists and VASARI-auto, with far higher
agreement between VASARI-auto methods. The time taken for neuroradiologists to
derive VASARI was substantially higher than VASARI-auto (mean time per case 317
vs. 3 seconds). A UK hospital workforce analysis forecast that three years of
VASARI featurisation would demand 29,777 consultant neuroradiologist workforce
hours ({\pounds}1,574,935), reducible to 332 hours of computing time (and
{\pounds}146 of power) with VASARI-auto. The best-performing survival model
utilised VASARI-auto features as opposed to those derived by neuroradiologists.
VASARI-auto is a highly efficient automated labelling system with equitable
performance across patient age or sex, a favourable economic profile if used as
a decision support tool, and with non-inferior fidelity in downstream patient
survival prediction. Future work should iterate upon and integrate such tools
to enhance patient care.

VASARI-auto 是一种高效的自动标记系统，无论患者年龄或性别，都具有公平的性能，如果用作决策支持工具，具有有利的经济效益，并具有不逊色的患者生存预测保真度。

VASARI-auto：平等、高效、经济的胶质瘤 MRI 特征提取

VASARI-auto: equitable, efficient, and economical featurisation of  glioma MRI

Most medical treatment decisions are sequential in nature. Hence, there is
substantial hope that reinforcement learning may make it possible to formulate
precise data-driven treatment plans. However, a key challenge for most
applications in this field is the sparse nature of primarily mortality-based
reward functions, leading to decreased stability of offline estimates. In this
work, we introduce a deep Q-learning approach able to obtain more reliable
critical care policies. This method integrates relevant but noisy intermediate
biomarker signals into the reward specification, without compromising the
optimization of the main outcome of interest (e.g. patient survival). We
achieve this by first pruning the action set based on all available rewards,
and second training a final model based on the sparse main reward but with a
restricted action set. By disentangling accurate and approximated rewards
through action pruning, potential distortions of the main objective are
minimized, all while enabling the extraction of valuable information from
intermediate signals that can guide the learning process. We evaluate our
method in both off-policy and offline settings using simulated environments and
real health records of patients in intensive care units. Our empirical results
indicate that pruning significantly reduces the size of the action space while
staying mostly consistent with the actions taken by physicians, outperforming
the current state-of-the-art offline reinforcement learning method conservative
Q-learning. Our work is a step towards developing reliable policies by
effectively harnessing the wealth of available information in data-intensive
critical care environments.

通过剪枝和训练最终模型两步，成功将中间生物标志信号融入到奖励范式中，在不损害主要结果的情况下，从中解出有价值的信息指导学习过程，从而在模拟环境和实际患者记录中成功开发了一种有效的离线强化学习方法，为开发可靠的策略迈出了一步。

经修剪多目标深度 Q 学习方法实现可靠的危重病护理政策

Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning  Approach to Critical Care

Cancer prognostication is a challenging task in computational pathology that
requires context-aware representations of histology features to adequately
infer patient survival. Despite the advancements made in weakly-supervised deep
learning, many approaches are not context-aware and are unable to model
important morphological feature interactions between cell identities and tissue
types that are prognostic for patient survival. In this work, we present
Patch-GCN, a context-aware, spatially-resolved patch-based graph convolutional
network that hierarchically aggregates instance-level histology features to
model local- and global-level topological structures in the tumor
microenvironment. We validate Patch-GCN with 4,370 gigapixel WSIs across five
different cancer types from the Cancer Genome Atlas (TCGA), and demonstrate
that Patch-GCN outperforms all prior weakly-supervised approaches by
3.58-9.46%. Our code and corresponding models are publicly available at
this https URL

本研究提出了一种 Patch-GCN 算法，该算法可以对肿瘤微环境中的局部和全局拓扑结构进行建模，从而对患者的生存预后进行更好的预测。该算法通过 4,370 张 WSIs 的验证，在 5 种不同的癌症类型中，表现出比之前研究中所有的弱监督方法都要好。

全切片图像是 2D 点云：基于补丁图卷积网络的上下文感知生存预测

Whole Slide Images are 2D Point Clouds: Context-Aware Survival  Prediction using Patch-based Graph Convolutional Networks

Gliomas are the most common primary brain malignancies, with different
degrees of aggressiveness, variable prognosis and various heterogeneous
histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic
core, active and non-enhancing core. This intrinsic heterogeneity is also
portrayed in their radio-phenotype, as their sub-regions are depicted by
varying intensity profiles disseminated across multi-parametric magnetic
resonance imaging (mpMRI) scans, reflecting varying biological properties.
Their heterogeneous shape, extent, and location are some of the factors that
make these tumors difficult to resect, and in some cases inoperable. The amount
of resected tumor is a factor also considered in longitudinal scans, when
evaluating the apparent tumor for potential diagnosis of progression.
Furthermore, there is mounting evidence that accurate segmentation of the
various tumor sub-regions can offer the basis for quantitative image analysis
towards prediction of patient overall survival. This study assesses the
state-of-the-art machine learning (ML) methods used for brain tumor image
analysis in mpMRI scans, during the last seven instances of the International
Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we
focus on i) evaluating segmentations of the various glioma sub-regions in
pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue
of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO
criteria, and iii) predicting the overall survival from pre-operative mpMRI
scans of patients that underwent gross total resection. Finally, we investigate
the challenge of identifying the best ML algorithms for each of these tasks,
considering that apart from being diverse on each instance of the challenge,
the multi-institutional mpMRI BraTS dataset has also been a continuously
evolving/growing dataset.

本研究评估了近年来机器学习方法在脑瘤 mpMRI 扫描方面的应用，主要集中在评估各种脑瘤子区域的分割、预测肿瘤进展和预测患者的总体存活率等方面，考虑到数据集的多样性和不断演化，同时探讨了针对每个任务的最佳机器学习算法的挑战。

用于脑肿瘤分割、进展评估和总体生存预测的最佳机器学习算法识别（BRATS 挑战）

Identifying the Best Machine Learning Algorithms for Brain Tumor  Segmentation, Progression Assessment, and Overall Survival Prediction in the  BRATS Challenge