Clinical variant classification of pathogenic versus benign genetic variants
remains a challenge in clinical genetics. Recently, the proposition of genomic
foundation models has improved the generic variant effect prediction (VEP)
accuracy via weakly-supervised or unsupervised training. However, these VEPs
are not disease-specific, limiting their adaptation at the point of care. To
address this problem, we propose DYNA: Disease-specificity fine-tuning via a
Siamese neural network broadly applicable to all genomic foundation models for
more effective variant effect predictions in disease-specific contexts. We
evaluate DYNA in two distinct disease-relevant tasks. For coding VEPs, we focus
on various cardiovascular diseases, where gene-disease relationships of
loss-of-function vs. gain-of-function dictate disease-specific VEP. For
non-coding VEPs, we apply DYNA to an essential post-transcriptional regulatory
axis of RNA splicing, the most common non-coding pathogenic mechanism in
established clinical VEP guidelines. In both cases, DYNA fine-tunes various
pre-trained genomic foundation models on small, rare variant sets. The DYNA
fine-tuned models show superior performance in the held-out rare variant
testing set and are further replicated in large, clinically-relevant variant
annotations in ClinVAR. Thus, DYNA offers a potent disease-specific variant
effect prediction method, excelling in intra-gene generalization and
generalization to unseen genetic variants, making it particularly valuable for
disease associations and clinical applicability.

通过提出 DYNA 方法，通过 Siamese 神经网络针对所有基因组学基础模型进行疾病特异性微调，从而提高了变异效应预测的准确性，这对疾病关联和临床适用性具有重要价值。

DYNA：用于变异致病性的疾病特定语言模型

DYNA: Disease-Specific Language Model for Variant Pathogenicity

Generative protein language models are a natural way to design new proteins
with desired functions. However, current models are either difficult to direct
to produce a protein from a specific family of interest, or must be trained on
a large multiple sequence alignment (MSA) from the specific family of interest,
making them unable to benefit from transfer learning across families. To
address this, we propose $\textbf{P}$r$\textbf{o}$tein $\textbf{E}$volutionary
$\textbf{T}$ransformer (PoET), an autoregressive generative model of whole
protein families that learns to generate sets of related proteins as
sequences-of-sequences across tens of millions of natural protein sequence
clusters. PoET can be used as a retrieval-augmented language model to generate
and score arbitrary modifications conditioned on any protein family of
interest, and can extrapolate from short context lengths to generalize well
even for small families. This is enabled by a unique Transformer layer; we
model tokens sequentially within sequences while attending between sequences
order invariantly, allowing PoET to scale to context lengths beyond those used
during training. PoET outperforms existing protein language models and
evolutionary sequence models for variant function prediction in extensive
experiments on deep mutational scanning datasets, improving variant effect
prediction across proteins of all MSA depths.

提出了一种全新的蛋白质进化转换器（PoET）, 它是一种自回归生成整个蛋白家族的模型，可以用作检索嵌入式语言模型，从而生成和评分任意修饰物，并对深度突变扫描数据集进行广泛实验，可以改善所有 MSA 深度的蛋白质变体效应预测。

PoET: 蛋白家族的生成模型，作为序列组成的序列

PoET: A generative model of protein families as sequences-of-sequences

Directed evolution plays an indispensable role in protein engineering that
revises existing protein sequences to attain new or enhanced functions.
Accurately predicting the effects of protein variants necessitates an in-depth
understanding of protein structure and function. Although large self-supervised
language models have demonstrated remarkable performance in zero-shot inference
using only protein sequences, these models inherently do not interpret the
spatial characteristics of protein structures, which are crucial for
comprehending protein folding stability and internal molecular interactions.
This paper introduces a novel pre-training framework that cascades sequential
and geometric analyzers for protein primary and tertiary structures. It guides
mutational directions toward desired traits by simulating natural selection on
wild-type proteins and evaluates the effects of variants based on their fitness
to perform the function. We assess the proposed approach using a public
database and two new databases for a variety of variant effect prediction
tasks, which encompass a diverse set of proteins and assays from different
taxa. The prediction results achieve state-of-the-art performance over other
zero-shot learning methods for both single-site mutations and deep mutations.

本文提出了一种新的预训练框架，它将用于蛋白质主体结构和三级结构的序列和几何分析器级联，模拟野生型蛋白质的自然选择并评估变体的效果，以指导遗传突变方向，实现所需特征，提出了一种新的变体影响预测方法，其预测结果在单点和深部变异方面表现良好。