Contrastive, self-supervised learning (SSL) is used to train a model that
predicts cancer type from miRNA, mRNA or RPPA expression data. This model, a
pretrained FT-Transformer, is shown to outperform XGBoost and CatBoost,
standard benchmarks for tabular data, when labelled samples are scarce but the
number of unlabelled samples is high. This is despite the fact that the
datasets we use have $\mathcal{O}(10^{1})$ classes and
$\mathcal{O}(10^{2})-\mathcal{O}(10^{4})$ features. After demonstrating the
efficacy of our chosen method of self-supervised pretraining, we investigate
SSL for multi-modal models. A late-fusion model is proposed, where each omics
is passed through its own sub-network, the outputs of which are averaged and
passed to the pretraining or downstream objective function. Multi-modal
pretraining is shown to improve predictions from a single omics, and we argue
that this is useful for datasets with many unlabelled multi-modal samples, but
few labelled unimodal samples. Additionally, we show that pretraining each
omics-specific module individually is highly effective. This enables the
application of the proposed model in a variety of contexts where a large amount
of unlabelled data is available from each omics, but only a few labelled
samples.

利用对比式自监督学习训练预训练的 FT-Transformer 模型，从 miRNA、mRNA 或 RPPA 表达数据中预测癌症类型。当标记样本稀缺而未标记样本众多时，相比标准的表格数据基准（XGBoost 和 CatBoost），该模型表现更好。我们还探讨了多模态模型的自监督学习，提出了一种后融合模型，其中每个组学通过自己的子网络进行处理，子网络输出求平均并传递给预训练或下游目标函数。实验证明多模态预训练可以提高单一组学的预测效果，并且对于未标记的多模态样本丰富而标记的单一组学样本稀缺的数据集，这是有用的。另外，我们还展示了分别对每个组学特定模块进行预训练的高效性。这使得该模型可以应用于大量未标记数据可用于每个组学的各种情境，但只有少数带标签样本的情况。

低标签、高数据情境下的多组学嵌入自监督学习

Self-supervised learning of multi-omics embeddings in the low-label,  high-data regime

Extracting genetic information from a full range of sequencing data is
important for understanding diseases. We propose a novel method to effectively
explore the landscape of genetic mutations and aggregate them to predict cancer
type. We used multinomial logistic regression, nonsmooth non-negative matrix
factorization (nsNMF), and support vector machine (SVM) to utilize the full
range of sequencing data, aiming at better aggregating genetic mutations and
improving their power in predicting cancer types. Specifically, we introduced a
classifier to distinguish cancer types using somatic mutations obtained from
whole-exome sequencing data. Mutations were identified from multiple cancers
and scored using SIFT, PP2, and CADD, and grouped at the individual gene level.
The nsNMF was then applied to reduce dimensionality and to obtain coefficient
and basis matrices. A feature matrix was derived from the obtained matrices to
train a classifier for cancer type classification with the SVM model. We have
demonstrated that the classifier was able to distinguish the cancer types with
reasonable accuracy. In five-fold cross-validations using mutation counts as
features, the average prediction accuracy was 77.1% (SEM=0.1%), significantly
outperforming baselines and outperforming models using mutation scores as
features. Using the factor matrices derived from the nsNMF, we identified
multiple genes and pathways that are significantly associated with each cancer
type. This study presents a generic and complete pipeline to study the
associations between somatic mutations and cancers. The discovered genes and
pathways associated with each cancer type can lead to biological insights. The
proposed method can be adapted to other studies for disease classification and
pathway discovery.

该研究提出了一种新的方法，利用多项逻辑回归、非平滑非负矩阵分解 (nsNMF) 和支持向量机 (SVM) 等算法，从全序列数据中提取遗传信息以更好地预测癌症类型。实验表明，该方法在突变计数、突变得分等方面均表现优异，可用于其他疾病分类和通路发现研究。