Recently, Large Language Models (LLMs) with their strong task-handling
capabilities have shown remarkable advancements across a spectrum of fields,
moving beyond natural language understanding. However, their proficiency within
the chemistry domain remains restricted, especially in solving professional
molecule-related tasks. This challenge is attributed to their inherent
limitations in comprehending molecules using only common textual
representations, i.e., SMILES strings. In this study, we seek to enhance the
ability of LLMs to comprehend molecules by designing and equipping them with a
multi-modal external module, namely MolX. In particular, instead of directly
using a SMILES string to represent a molecule, we utilize specific encoders to
extract fine-grained features from both SMILES string and 2D molecular graph
representations for feeding into an LLM. Moreover, a human-defined molecular
fingerprint is incorporated to leverage its embedded domain knowledge. Then, to
establish an alignment between MolX and the LLM's textual input space, the
whole model in which the LLM is frozen, is pre-trained with a versatile
strategy including a diverse set of tasks. Extensive experimental evaluations
demonstrate that our proposed method only introduces a small number of
trainable parameters while outperforming baselines on various downstream
molecule-related tasks ranging from molecule-to-text translation to
retrosynthesis, with and without fine-tuning the LLM.

本研究设计和装备 MolX 作为一种多模态外部模块，通过使用特定编码器从 SMILES 字符串和 2D 分子图表示中提取细粒度特征，从而增强了大型语言模型（LLMs）在理解分子方面的能力。实验结果表明，我们提出的方法在多种分子相关任务中表现出色，包括从分子到文本的翻译和分子的逆合成，无论是否对 LLM 进行微调。

MolX: 使用多模态扩展增强大型语言模型在分子学习中的能力

MolX: Enhancing Large Language Models for Molecular Learning with A  Multi-Modal Extension

We present $\textbf{MolT5}$ $-$ a self-supervised learning framework for
pretraining models on a vast amount of unlabeled natural language text and
molecule strings. $\textbf{MolT5}$ allows for new, useful, and challenging
analogs of traditional vision-language tasks, such as molecule captioning and
text-based de novo molecule generation (altogether: translation between
molecules and language), which we explore for the first time. Since
$\textbf{MolT5}$ pretrains models on single-modal data, it helps overcome the
chemistry domain shortcoming of data scarcity. Furthermore, we consider several
metrics, including a new cross-modal embedding-based metric, to evaluate the
tasks of molecule captioning and text-based molecule generation. Our results
show that $\textbf{MolT5}$-based models are able to generate outputs, both
molecules and captions, which in many cases are high quality.

$\textbf {MolT5}$ 是一个自监督学习框架，可用于对大量未标记的自然语言文本和分子字符串进行预训练，允许新的、有用的和具有挑战性的类比于传统的视觉语言任务。