Protein language models have demonstrated significant potential in the field
of protein engineering. However, current protein language models primarily
operate at the residue scale, which limits their ability to provide information
at the atom level. This limitation prevents us from fully exploiting the
capabilities of protein language models for applications involving both
proteins and small molecules. In this paper, we propose ms-ESM (multi-scale
ESM), a novel approach that enables multi-scale unified molecular modeling.
ms-ESM achieves this by pre-training on multi-scale code-switch protein
sequences and utilizing a multi-scale position encoding to capture
relationships among residues and atoms. Experimental results indicate that
ms-ESM surpasses previous methods in protein-molecule tasks, demonstrating the
full utilization of protein language models. Further investigations reveal that
through unified molecular modeling, ms-ESM not only gains molecular knowledge
but also retains its understanding of proteins.

通过在多尺度编码切换的蛋白质序列上进行预训练，并利用多尺度位置编码来捕捉残基和原子之间的关系，ms-ESM（多尺度 ESM）提出了一种新颖的方法，实现了多尺度统一的分子建模，实验结果表明，ms-ESM 在蛋白质 - 分子任务方面超过了以前的方法，展示了蛋白质语言模型的全部利用，进一步的研究发现，通过统一的分子建模，ms-ESM 不仅获得了分子知识，还保留了对蛋白质的理解。