Multiple Sequence Alignment (MSA) plays a pivotal role in unveiling the
evolutionary trajectories of protein families. The accuracy of protein
structure predictions is often compromised for protein sequences that lack
sufficient homologous information to construct high quality MSA. Although
various methods have been proposed to generate virtual MSA under these
conditions, they fall short in comprehensively capturing the intricate
coevolutionary patterns within MSA or require guidance from external oracle
models. Here we introduce MSAGPT, a novel approach to prompt protein structure
predictions via MSA generative pretraining in the low MSA regime. MSAGPT
employs a simple yet effective 2D evolutionary positional encoding scheme to
model complex evolutionary patterns. Endowed by this, its flexible 1D MSA
decoding framework facilitates zero or few shot learning. Moreover, we
demonstrate that leveraging the feedback from AlphaFold2 can further enhance
the model capacity via Rejective Fine tuning (RFT) and Reinforcement Learning
from AF2 Feedback (RLAF). Extensive experiments confirm the efficacy of MSAGPT
in generating faithful virtual MSA to enhance the structure prediction
accuracy. The transfer learning capabilities also highlight its great potential
for facilitating other protein tasks.

MSAGPT 是一种新颖的方法，通过在低 MSA 水平下进行 MSA 生成预训练来促进蛋白质结构预测，其通过简单而有效的 2D 进化位置编码方案来建模复杂的进化模式，并通过 AlphaFold2 的反馈进一步增强模型能力，证实了 MSAGPT 在生成忠实的虚拟 MSA 以提高结构预测准确性方面的功效，并突显了其促进其他蛋白质任务的巨大潜力。

MSAGPT: 利用 MSA 生成预训练进行神经启发蛋白质结构预测

MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative  Pre-Training

We present the MSA-to-protein transformer, a generative model of protein
sequences conditioned on protein families represented by multiple sequence
alignments (MSAs). Unlike existing approaches to learning generative models of
protein families, the MSA-to-protein transformer conditions sequence generation
directly on a learned encoding of the multiple sequence alignment,
circumventing the need for fitting dedicated family models. By training on a
large set of well-curated multiple sequence alignments in Pfam, our
MSA-to-protein transformer generalizes well to protein families not observed
during training and outperforms conventional family modeling approaches,
especially when MSAs are small. Our generative approach accurately models
epistasis and indels and allows for exact inference and efficient sampling
unlike other approaches. We demonstrate the protein sequence modeling
capabilities of our MSA-to-protein transformer and compare it with alternative
sequence modeling approaches in comprehensive benchmark experiments.

本文提出了基于 MSA 的蛋白质变换器，是一种基于多序列比对的蛋白质序列生成模型，能够准确地建模表观效应和插入缺失，并且相比已有的基于专用家族模型的方法，其泛化性能更好，尤其在 MSAs 很小的情况下表现出色，能够精确推断和高效采样。