Molecular property prediction with deep learning has gained much attention
over the past years. Owing to the scarcity of labeled molecules, there has been
growing interest in self-supervised learning methods that learn generalizable
molecular representations from unlabeled data. Molecules are typically treated
as 2D topological graphs in modeling, but it has been discovered that their 3D
geometry is of great importance in determining molecular functionalities. In
this paper, we propose the Geometry-aware line graph transformer (Galformer)
pre-training, a novel self-supervised learning framework that aims to enhance
molecular representation learning with 2D and 3D modalities. Specifically, we
first design a dual-modality line graph transformer backbone to encode the
topological and geometric information of a molecule. The designed backbone
incorporates effective structural encodings to capture graph structures from
both modalities. Then we devise two complementary pre-training tasks at the
inter and intra-modality levels. These tasks provide properly supervised
information and extract discriminative 2D and 3D knowledge from unlabeled
molecules. Finally, we evaluate Galformer against six state-of-the-art
baselines on twelve property prediction benchmarks via downstream fine-tuning.
Experimental results show that Galformer consistently outperforms all baselines
on both classification and regression tasks, demonstrating its effectiveness.

本文提出了 Geometry-aware line graph transformer (Galformer) 预训练方法，采用 2D 和 3D 模态增强分子表示学习，通过对无标签的分子进行两个互补的预训练任务，提取出 2D 和 3D 知识，并在性质预测方面表现出优越性能。