Predicting which proteins interact together from amino-acid sequences is an
important task. We develop a method to pair interacting protein sequences which
leverages the power of protein language models trained on multiple sequence
alignments, such as MSA Transformer and the EvoFormer module of AlphaFold. We
formulate the problem of pairing interacting partners among the paralogs of two
protein families in a differentiable way. We introduce a method called DiffPALM
that solves it by exploiting the ability of MSA Transformer to fill in masked
amino acids in multiple sequence alignments using the surrounding context. MSA
Transformer encodes coevolution between functionally or structurally coupled
amino acids. We show that it captures inter-chain coevolution, while it was
trained on single-chain data, which means that it can be used
out-of-distribution. Relying on MSA Transformer without fine-tuning, DiffPALM
outperforms existing coevolution-based pairing methods on difficult benchmarks
of shallow multiple sequence alignments extracted from ubiquitous prokaryotic
protein datasets. It also outperforms an alternative method based on a
state-of-the-art protein language model trained on single sequences. Paired
alignments of interacting protein sequences are a crucial ingredient of
supervised deep learning methods to predict the three-dimensional structure of
protein complexes. DiffPALM substantially improves the structure prediction of
some eukaryotic protein complexes by AlphaFold-Multimer, without significantly
deteriorating any of those we tested. It also achieves competitive performance
with using orthology-based pairing.

从蛋白质序列中预测相互作用的蛋白质，并使用蛋白质语言模型的先进方法进行匹配。

使用掩码语言建模配对相互作用蛋白序列

Pairing interacting protein sequences using masked language modeling

Multiple sequence alignments (MSAs) of proteins encode rich biological
information and have been workhorses in bioinformatic methods for tasks like
protein design and protein structure prediction for decades. Recent
breakthroughs like AlphaFold2 that use transformers to attend directly over
large quantities of raw MSAs have reaffirmed their importance. Generation of
MSAs is highly computationally intensive, however, and no datasets comparable
to those used to train AlphaFold2 have been made available to the research
community, hindering progress in machine learning for proteins. To remedy this
problem, we introduce OpenProteinSet, an open-source corpus of more than 16
million MSAs, associated structural homologs from the Protein Data Bank, and
AlphaFold2 protein structure predictions. We have previously demonstrated the
utility of OpenProteinSet by successfully retraining AlphaFold2 on it. We
expect OpenProteinSet to be broadly useful as training and validation data for
1) diverse tasks focused on protein structure, function, and design and 2)
large-scale multimodal machine learning research.

为了推动蛋白质的机器学习研究进展，我们介绍了一个开源数据集 OpenProteinSet，其中包含超过 1600 万个蛋白质多序列比对，与蛋白质数据库中的结构同源物和 AlphaFold2 蛋白质结构预测相对应，可广泛用于蛋白质结构、功能、设计以及大规模多模式机器学习研究的训练和验证数据。

OpenProteinSet: 大规模结构生物学训练数据

OpenProteinSet: Training data for structural biology at scale

AI-based protein structure prediction pipelines, such as AlphaFold2, have
achieved near-experimental accuracy. These advanced pipelines mainly rely on
Multiple Sequence Alignments (MSAs) as inputs to learn the co-evolution
information from the homologous sequences. Nonetheless, searching MSAs from
protein databases is time-consuming, usually taking dozens of minutes.
Consequently, we attempt to explore the limits of fast protein structure
prediction by using only primary sequences of proteins. HelixFold-Single is
proposed to combine a large-scale protein language model with the superior
geometric learning capability of AlphaFold2. Our proposed method,
HelixFold-Single, first pre-trains a large-scale protein language model (PLM)
with thousands of millions of primary sequences utilizing the self-supervised
learning paradigm, which will be used as an alternative to MSAs for learning
the co-evolution information. Then, by combining the pre-trained PLM and the
essential components of AlphaFold2, we obtain an end-to-end differentiable
model to predict the 3D coordinates of atoms from only the primary sequence.
HelixFold-Single is validated in datasets CASP14 and CAMEO, achieving
competitive accuracy with the MSA-based methods on the targets with large
homologous families. Furthermore, HelixFold-Single consumes much less time than
the mainstream pipelines for protein structure prediction, demonstrating its
potential in tasks requiring many predictions. The code of HelixFold-Single is
available at
this https URL,
and we also provide stable web services on
this https URL.

本研究提出了一种新的蛋白质结构预测方法 ——HelixFold-Single，它基于大规模蛋白质语言模型，将原始序列代替同源序列用于学习蛋白质序列的共同进化信息，并结合 AlphaFold2 的关键组件预测蛋白质原子的三维坐标，其在 CASP14 和 CAMEO 数据集中准确度表现良好，同时能够节省大量的计算时间。

HelixFold-Single: 使用蛋白语言模型作为替代品进行无多序列比对蛋白质结构预测

HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative

We present the MSA-to-protein transformer, a generative model of protein
sequences conditioned on protein families represented by multiple sequence
alignments (MSAs). Unlike existing approaches to learning generative models of
protein families, the MSA-to-protein transformer conditions sequence generation
directly on a learned encoding of the multiple sequence alignment,
circumventing the need for fitting dedicated family models. By training on a
large set of well-curated multiple sequence alignments in Pfam, our
MSA-to-protein transformer generalizes well to protein families not observed
during training and outperforms conventional family modeling approaches,
especially when MSAs are small. Our generative approach accurately models
epistasis and indels and allows for exact inference and efficient sampling
unlike other approaches. We demonstrate the protein sequence modeling
capabilities of our MSA-to-protein transformer and compare it with alternative
sequence modeling approaches in comprehensive benchmark experiments.

本文提出了基于 MSA 的蛋白质变换器，是一种基于多序列比对的蛋白质序列生成模型，能够准确地建模表观效应和插入缺失，并且相比已有的基于专用家族模型的方法，其泛化性能更好，尤其在 MSAs 很小的情况下表现出色，能够精确推断和高效采样。