Protein representation learning aims to learn informative protein embeddings
capable of addressing crucial biological questions, such as protein function
prediction. Although sequence-based transformer models have shown promising
results by leveraging the vast amount of protein sequence data in a
self-supervised way, there is still a gap in applying these methods to 3D
protein structures. In this work, we propose a pre-training scheme going beyond
trivial masking methods leveraging 3D and hierarchical structures of proteins.
We propose a novel self-supervised method to pretrain 3D graph neural networks
on 3D protein structures, by predicting the distances between local geometric
centroids of protein subgraphs and the global geometric centroid of the
protein. The motivation for this method is twofold. First, the relative spatial
arrangements and geometric relationships among different regions of a protein
are crucial for its function. Moreover, proteins are often organized in a
hierarchical manner, where smaller substructures, such as secondary structure
elements, assemble into larger domains. By considering subgraphs and their
relationships to the global protein structure, the model can learn to reason
about these hierarchical levels of organization. We experimentally show that
our proposed pertaining strategy leads to significant improvements in the
performance of 3D GNNs in various protein classification tasks.

本研究通过预训练三维蛋白质结构的图神经网络，利用蛋白质的地理层次结构和亚图与全球结构的关系，提出了一种新的自我监督方法，能够显著提高三维图神经网络在不同蛋白质分类任务中的性能表现。

利用子图在 3D 蛋白质结构上进行几何自监督预训练

Geometric Self-Supervised Pretraining on 3D Protein Structures using  Subgraphs

Learning from 3D protein structures has gained wide interest in protein
modeling and structural bioinformatics. Unfortunately, the number of available
structures is orders of magnitude lower than the training data sizes commonly
used in computer vision and machine learning. Moreover, this number is reduced
even further, when only annotated protein structures can be considered, making
the training of existing models difficult and prone to over-fitting. To address
this challenge, we introduce a new representation learning framework for 3D
protein structures. Our framework uses unsupervised contrastive learning to
learn meaningful representations of protein structures, making use of proteins
from the Protein Data Bank. We show, how these representations can be used to
solve a large variety of tasks, such as protein function prediction, protein
fold classification, structural similarity prediction, and protein-ligand
binding affinity prediction. Moreover, we show how fine-tuned networks,
pre-trained with our algorithm, lead to significantly improved task
performance, achieving new state-of-the-art results in many tasks.

本文介绍了一种利用无监督对比学习来学习有意义的蛋白质结构表示的框架，其利用蛋白质数据银行中的蛋白质，展示了这些表示可以用于解决多种任务，如蛋白质功能预测、蛋白质折叠分类、结构相似性预测和蛋白质配体结合亲和力预测。此外，作者还表明，使用该算法预训练的微调网络在许多任务中取得了新的最优结果。