Language Models (LMs) have greatly influenced diverse domains. However, their
inherent limitation in comprehending 3D molecular structures has considerably
constrained their potential in the biomolecular domain. To bridge this gap, we
focus on 3D molecule-text interpretation, and propose 3D-MoLM: 3D-Molecular
Language Modeling. Specifically, 3D-MoLM enables an LM to interpret and analyze
3D molecules by equipping the LM with a 3D molecular encoder. This integration
is achieved by a 3D molecule-text projector, bridging the 3D molecular
encoder's representation space and the LM's input space. Moreover, to enhance
3D-MoLM's ability of cross-modal molecular understanding and instruction
following, we meticulously curated a 3D molecule-centric instruction tuning
dataset -- 3D-MoIT. Through 3D molecule-text alignment and 3D molecule-centric
instruction tuning, 3D-MoLM establishes an integration of 3D molecular encoder
and LM. It significantly surpasses existing baselines on downstream tasks,
including molecule-text retrieval, molecule captioning, and more challenging
open-text molecular QA tasks, especially focusing on 3D-dependent properties.

通过在语言模型中嵌入 3D 分子编码器，我们提出了 3D-MoLM：3D 分子语言建模，用于解决语言模型在理解三维分子结构方面的限制，在生物分子领域具有潜力。