Pathology reports are rich in clinical and pathological details but are often
presented in free-text format. The unstructured nature of these reports
presents a significant challenge limiting the accessibility of their content.
In this work, we present a practical approach based on the use of large
multimodal models (LMMs) for automatically extracting information from scanned
images of pathology reports with the goal of generating a standardised report
specifying the value of different fields along with estimated confidence about
the accuracy of the extracted fields. The proposed approach overcomes
limitations of existing methods which do not assign confidence scores to
extracted fields limiting their practical use. The proposed framework uses two
stages of prompting a Large Multimodal Model (LMM) for information extraction
and validation. The framework generalises to textual reports from multiple
medical centres as well as scanned images of legacy pathology reports. We show
that the estimated confidence is an effective indicator of the accuracy of the
extracted information that can be used to select only accurately extracted
fields. We also show the prognostic significance of structured and unstructured
data from pathology reports and show that the automatically extracted field
values significant prognostic value for patient stratification. The framework
is available for evaluation via the URL: this https URL

使用大型多模型自动从病理报告图片中提取信息，并生成标准化报告，其中包括不同字段的数值及其准确性的置信度。提取的字段值在病理报告中具有重要的预后价值，可用于患者分层。

基于大型多模态模型的病理报告标准化与置信度及其预后意义

Large Multimodal Model based Standardisation of Pathology Reports with  Confidence and their Prognostic Significance

In the realm of precision medicine, effective patient stratification and
disease subtyping demand innovative methodologies tailored for multi-omics
data. Clustering techniques applied to multi-omics data have become
instrumental in identifying distinct subgroups of patients, enabling a
finer-grained understanding of disease variability. This work establishes a
powerful framework for advancing precision medicine through unsupervised
random-forest-based clustering and federated computing. We introduce a novel
multi-omics clustering approach utilizing unsupervised random-forests. The
unsupervised nature of the random forest enables the determination of
cluster-specific feature importance, unraveling key molecular contributors to
distinct patient groups. Moreover, our methodology is designed for federated
execution, a crucial aspect in the medical domain where privacy concerns are
paramount. We have validated our approach on machine learning benchmark data
sets as well as on cancer data from The Cancer Genome Atlas (TCGA). Our method
is competitive with the state-of-the-art in terms of disease subtyping, but at
the same time substantially improves the cluster interpretability. Experiments
indicate that local clustering performance can be improved through federated
computing.

通过无监督随机森林聚类和联合计算，建立了一个强大的框架来推进精准医学，实现有效的患者分层和疾病亚型划分，改进了疾病亚型的解释性，并优化了局部聚类性能。

隐私保护患者分层的联合非监督随机森林

Federated unsupervised random forest for privacy-preserving patient  stratification

With the long-term rapid increase in incidences of colorectal cancer (CRC),
there is an urgent clinical need to improve risk stratification. The
conventional pathology report is usually limited to only a few
histopathological features. However, most of the tumor microenvironments used
to describe patterns of aggressive tumor behavior are ignored. In this work, we
aim to learn histopathological patterns within cancerous tissue regions that
can be used to improve prognostic stratification for colorectal cancer. To do
so, we propose a self-supervised learning method that jointly learns a
representation of tissue regions as well as a metric of the clustering to
obtain their underlying patterns. These histopathological patterns are then
used to represent the interaction between complex tissues and predict clinical
outcomes directly. We furthermore show that the proposed approach can benefit
from linear predictors to avoid overfitting in patient outcomes predictions. To
this end, we introduce a new well-characterized clinicopathological dataset,
including a retrospective collective of 374 patients, with their survival time
and treatment information. Histomorphological clusters obtained by our method
are evaluated by training survival models. The experimental results demonstrate
statistically significant patient stratification, and our approach outperformed
the state-of-the-art deep clustering methods.

本研究旨在通过自我监督学习方法，学习肿瘤组织内的组织病理模式，以优化结直肠癌的预后分层，并通过新的数据集、患者预后模型和实验证明了其效果优于现有深度聚类方法。