Molecular representation learning is fundamental for many drug related
applications. Most existing molecular pre-training models are limited in using
single molecular modality, either SMILES or graph representation. To
effectively leverage both modalities, we argue that it is critical to capture
the fine-grained 'semantics' between SMILES and graph, because subtle
sequence/graph differences may lead to contrary molecular properties. In this
paper, we propose a universal SMILE-graph representation learning model, namely
UniMAP. Firstly, an embedding layer is employed to obtain the token and
node/edge representation in SMILES and graph, respectively. A multi-layer
Transformer is then utilized to conduct deep cross-modality fusion. Specially,
four kinds of pre-training tasks are designed for UniMAP, including Multi-Level
Cross-Modality Masking (CMM), SMILES-Graph Matching (SGM), Fragment-Level
Alignment (FLA), and Domain Knowledge Learning (DKL). In this way, both global
(i.e. SGM and DKL) and local (i.e. CMM and FLA) alignments are integrated to
achieve comprehensive cross-modality fusion. We evaluate UniMAP on various
downstream tasks, i.e. molecular property prediction, drug-target affinity
prediction and drug-drug interaction. Experimental results show that UniMAP
outperforms current state-of-the-art pre-training methods.We also visualize the
learned representations to demonstrate the effect of multi-modality
integration.

本文提出了一种称为 UniMAP 的通用 SMILE-graph 表示学习模型，通过底层嵌入层和多层 Transformer 的使用，实现了跨多模态融合，包括多级跨模态掩码、SMILES-Graph 匹配、片段级对齐和领域知识学习等预训练任务，通过在多种下游任务上的实验证明了 UniMAP 在分子性质预测、药物 - 目标亲和性预测以及药物 - 药物相互作用方面的优越性能。同时展示了学习到的表示在多模态集成中的效果。