BriefGPT.xyz
Feb, 2025
能学习关注欧几里得距离的蛋白质训练变换器
Transformers trained on proteins can learn to attend to Euclidean distance
HTML
PDF
Isaac Ellmen, Constantin Schneider, Matthew I. J. Raybould, Charlotte M. Deane
TL;DR
本研究解决了变换器在蛋白质结构建模中的应用不足问题。论文提出了一种新颖的方法,通过线性坐标嵌入,变换器能够独立作为结构模型进行学习。研究发现,预训练的蛋白质变换器编码器在下游任务中的表现超过了定制的结构模型,展示了使用标准变换器作为混合结构-语言模型的潜力。
Abstract
While conventional
Transformers
generally operate on sequence data, they can be used in conjunction with structure models, typically SE(3)-invariant or equivariant graph neural networks (GNNs), for 3D applications such as
→