Pretraining molecular representation models without labels is fundamental to
various applications. Conventional methods mainly process 2D molecular graphs
and focus solely on 2D tasks, making their pretrained models incapable of
characterizing 3D geometry and thus defective for downstream 3D tasks. In this
work, we tackle 3D molecular pretraining in a complete and novel sense. In
particular, we first propose to adopt an equivariant energy-based model as the
backbone for pretraining, which enjoys the merits of fulfilling the symmetry of
3D space. Then we develop a node-level pretraining loss for force prediction,
where we further exploit the Riemann-Gaussian distribution to ensure the loss
to be E(3)-invariant, enabling more robustness. Moreover, a graph-level noise
scale prediction task is also leveraged to further promote the eventual
performance. We evaluate our model pretrained from a large-scale 3D dataset
GEOM-QM9 on two challenging 3D benchmarks: MD17 and QM9. Experimental results
demonstrate the efficacy of our method against current state-of-the-art
pretraining approaches, and verify the validity of our design for each proposed
component.

本文提出一种新的三维分子预训练方法，该方法采用等变能量基模型作为主干网络，以预测节点力信息为目标进行节点级预训练损失计算，并采用高斯分布以保证其 E（3）不变性，同时通过图层级噪声缩放预测任务来进一步提高模型性能，实验结果表明，该方法在两个挑战性的三维基准测试上均取得了比当前最先进的预训练方法更佳的表现。