In recent years, self-supervised learning has emerged as a powerful tool to
harness abundant unlabelled data for representation learning and has been
broadly adopted in diverse areas. However, when applied to molecular
representation learning (MRL), prevailing techniques such as masked sub-unit
reconstruction often fall short, due to the high degree of freedom in the
possible combinations of atoms within molecules, which brings insurmountable
complexity to the masking-reconstruction paradigm. To tackle this challenge, we
introduce REMO, a self-supervised learning framework that takes advantage of
well-defined atom-combination rules in common chemistry. Specifically, REMO
pre-trains graph/Transformer encoders on 1.7 million known chemical reactions
in the literature. We propose two pre-training objectives: Masked Reaction
Centre Reconstruction (MRCR) and Reaction Centre Identification (RCI). REMO
offers a novel solution to MRL by exploiting the underlying shared patterns in
chemical reactions as \textit{context} for pre-training, which effectively
infers meaningful representations of common chemistry knowledge. Such
contextual representations can then be utilized to support diverse downstream
molecular tasks with minimum finetuning, such as affinity prediction and
drug-drug interaction prediction. Extensive experimental results on
MoleculeACE, ACNet, drug-drug interaction (DDI), and reaction type
classification show that across all tested downstream tasks, REMO outperforms
the standard baseline of single-molecule masked modeling used in current MRL.
Remarkably, REMO is the pioneering deep learning model surpassing
fingerprint-based methods in activity cliff benchmarks.

借助自监督学习的 REM0 框架，基于化学反应的上下文，对分子表示学习进行了预训练，提供了富含化学知识的有意义的上下文表示，用于支持各种下游分子任务，同时在活性分析基准中超过了指纹方法。