The integration of molecule and language has garnered increasing attention in
molecular science. Recent advancements in Language Models (LMs) have
demonstrated potential for the comprehensive modeling of molecule and language.
However, existing works exhibit notable limitations. Most existing works
overlook the modeling of 3D information, which is crucial for understanding
molecular structures and also functions. While some attempts have been made to
leverage external structure encoding modules to inject the 3D molecular
information into LMs, there exist obvious difficulties that hinder the
integration of molecular structure and language text, such as modality
alignment and separate tuning. To bridge this gap, we propose 3D-MolT5, a
unified framework designed to model both 1D molecular sequence and 3D molecular
structure. The key innovation lies in our methodology for mapping fine-grained
3D substructure representations (based on 3D molecular fingerprints) to a
specialized 3D token vocabulary for 3D-MolT5. This 3D structure token
vocabulary enables the seamless combination of 1D sequence and 3D structure
representations in a tokenized format, allowing 3D-MolT5 to encode molecular
sequence (SELFIES), molecular structure, and text sequences within a unified
architecture. Alongside, we further introduce 1D and 3D joint pre-training to
enhance the model's comprehension of these diverse modalities in a joint
representation space and better generalize to various tasks for our foundation
model. Through instruction tuning on multiple downstream datasets, our proposed
3D-MolT5 shows superior performance than existing methods in molecular property
prediction, molecule captioning, and text-based molecule generation tasks. Our
code will be available on GitHub soon.

我们提出了 3D-MolT5 框架，通过在特定的三维令牌词汇表上映射细粒度的三维亚结构表示，实现了一维分子序列和三维分子结构的无缝组合，在一个统一的架构中编码了分子序列、分子结构和文本序列。对于分子属性预测、分子说明和基于文本的分子生成任务，我们的 3D-MolT5 在多个下游数据集上展现出优异的性能。