The ability to accurately model the fitness landscape of protein sequences is
critical to a wide range of applications, from quantifying the effects of human
variants on disease likelihood, to predicting immune-escape mutations in
viruses and designing novel biotherapeutic proteins. Deep generative models of
protein sequences trained on multiple sequence alignments have been the most
successful approaches so far to address these tasks. The performance of these
methods is however contingent on the availability of sufficiently deep and
diverse alignments for reliable training. Their potential scope is thus limited
by the fact many protein families are hard, if not impossible, to align. Large
language models trained on massive quantities of non-aligned protein sequences
from diverse families address these problems and show potential to eventually
bridge the performance gap. We introduce Tranception, a novel transformer
architecture leveraging autoregressive predictions and retrieval of homologous
sequences at inference to achieve state-of-the-art fitness prediction
performance. Given its markedly higher performance on multiple mutants,
robustness to shallow alignments and ability to score indels, our approach
offers significant gain of scope over existing approaches. To enable more
rigorous model testing across a broader range of protein families, we develop
ProteinGym -- an extensive set of multiplexed assays of variant effects,
substantially increasing both the number and diversity of assays compared to
existing benchmarks.

介绍了使用 Tranception 模型和 ProteinGym 数据集实现蛋白质序列的健康预测的方法。