Proteins are essential to life's processes, underpinning evolution and
diversity. Advances in sequencing technology have revealed millions of
proteins, underscoring the need for sophisticated pre-trained protein models
for biological analysis and AI development. Facebook's ESM2, the most advanced
protein language model to date, leverages a masked prediction task for
unsupervised learning, crafting amino acid representations with notable
biochemical accuracy. Yet, it lacks in delivering functional protein insights,
signaling an opportunity for enhancing representation quality.Our study
addresses this gap by incorporating protein family classification into ESM2's
training.This approach, augmented with Community Propagation-Based Clustering
Algorithm, improves global protein representations, while a contextual
prediction task fine-tunes local amino acid accuracy. Significantly, our model
achieved state-of-the-art results in several downstream experiments,
demonstrating the power of combining global and local methodologies to
substantially boost protein representation quality.

本研究使用 ESM2 最先进的蛋白质语言模型，并结合蛋白质家族分类和基于社区传播的聚类算法，改善了全局蛋白质表示和局部氨基酸准确性，从而大幅提升蛋白质表达质量，并在多个下游实验中取得了最先进的结果。