Large-scale Protein Language Models (PLMs) have improved performance in
protein prediction tasks, ranging from 3D structure prediction to various
function predictions. In particular, AlphaFold, a ground-breaking AI system,
could potentially reshape structural biology. However, the utility of the PLM
module in AlphaFold, Evoformer, has not been explored beyond structure
prediction. In this paper, we investigate the representation ability of three
popular PLMs: ESM-1b (single sequence), MSA-Transformer (multiple sequence
alignment) and Evoformer (structural), with a special focus on Evoformer.
Specifically, we aim to answer the following key questions: (i) Does the
Evoformer trained as part of AlphaFold produce representations amenable to
predicting protein function? (ii) If yes, can Evoformer replace ESM-1b and
MSA-Transformer? (ii) How much do these PLMs rely on evolution-related protein
data? In this regard, are they complementary to each other? We compare these
models by empirical study along with new insights and conclusions. All code and
datasets for reproducibility are available at
this https URL.

本文研究探讨了 ESM-1b、MSA-Transformer 和 Evoformer 等三种常用蛋白质语言模型的表征能力，重点关注 Evoformer 在蛋白功能预测方面的表现，旨在回答三个关键问题：（i）AlphaFold 中的 Evoformer 能否产生适合预测蛋白功能的表征？（ii）如果可以，Evoformer 能否取代 ESM-1b 和 MSA-Transformer？（iii）这些 PLMs 在多大程度上依赖于与进化有关的蛋白数据？在这方面，它们是否互补？通过实证研究和新的见解和结论进行比较。附有完整的代码和数据集，可供复现。