Predicting genetic mutations from whole slide images is indispensable for
cancer diagnosis. However, existing work training multiple binary
classification models faces two challenges: (a) Training multiple binary
classifiers is inefficient and would inevitably lead to a class imbalance
problem. (b) The biological relationships among genes are overlooked, which
limits the prediction performance. To tackle these challenges, we innovatively
design a Biological-knowledge enhanced PathGenomic multi-label Transformer to
improve genetic mutation prediction performances. BPGT first establishes a
novel gene encoder that constructs gene priors by two carefully designed
modules: (a) A gene graph whose node features are the genes' linguistic
descriptions and the cancer phenotype, with edges modeled by genes' pathway
associations and mutation consistencies. (b) A knowledge association module
that fuses linguistic and biomedical knowledge into gene priors by
transformer-based graph representation learning, capturing the intrinsic
relationships between different genes' mutations. BPGT then designs a label
decoder that finally performs genetic mutation prediction by two tailored
modules: (a) A modality fusion module that firstly fuses the gene priors with
critical regions in WSIs and obtains gene-wise mutation logits. (b) A
comparative multi-label loss that emphasizes the inherent comparisons among
mutation status to enhance the discrimination capabilities. Sufficient
experiments on The Cancer Genome Atlas benchmark demonstrate that BPGT
outperforms the state-of-the-art.

从全幻灯图像中预测基因突变对于癌症诊断至关重要。本研究设计了一种生物知识增强的 PathGenomic 多标签 Transformer，通过构建基因优先级和基因突变预测模块，提高了预测性能，实验证明该方法优于现有最先进技术。

通过生物医学 - 语言知识增强的多标签分类预测基因突变从全切片图像中

Predicting Genetic Mutation from Whole Slide Images via  Biomedical-Linguistic Knowledge Enhanced Multi-label Classification

Whole slide imaging is fundamental to biomedical microscopy and computational
pathology. However, whole slide images (WSIs) present a complex computer vision
challenge due to their gigapixel size, diverse histopathologic features,
spatial heterogeneity, and limited/absent data annotations. These challenges
highlight that supervised training alone can result in suboptimal whole slide
representations. Self-supervised representation learning can achieve
high-quality WSI visual feature learning for downstream diagnostic tasks, such
as cancer diagnosis or molecular genetic prediction. Here, we present a general
self-supervised whole slide learning (S3L) framework for gigapixel-scale
self-supervision of WSIs. S3L combines data transformation strategies from
transformer-based vision and language modeling into a single unified framework
to generate paired views for self-supervision. S3L leverages the inherent
regional heterogeneity, histologic feature variability, and information
redundancy within WSIs to learn high-quality whole-slide representations. We
benchmark S3L visual representations on two diagnostic tasks for two biomedical
microscopy modalities. S3L significantly outperforms WSI baselines for cancer
diagnosis and genetic mutation prediction. Additionally, S3L achieves good
performance using both in-domain and out-of-distribution patch encoders,
demonstrating good flexibility and generalizability.

自我监督整个切片学习（S3L）框架可以通过利用整个切片的固有区域异质性、组织学特征变异性和信息冗余性来学习高质量的整个切片表示，用于癌症诊断和基因突变预测等诊断任务，并且在两种生物医学显微镜条件下显著优于基线模型。