AI for drug discovery has been a research hotspot in recent years, and
SMILES-based language models has been increasingly applied in drug molecular
design. However, no work has explored whether and how language models
understand the chemical spatial structure from 1D sequences. In this work, we
pre-train a transformer model on chemical language and fine-tune it toward drug
design objectives, and investigate the correspondence between high-frequency
SMILES substrings and molecular fragments. The results indicate that language
models can understand chemical structures from the perspective of molecular
fragments, and the structural knowledge learned through fine-tuning is
reflected in the high-frequency SMILES substrings generated by the model.

AI 用于药物发现是近年来的研究热点，基于 SMILES 的语言模型越来越多地应用于药物分子设计。本研究探讨了语言模型是否以及如何从 1D 序列中理解化学空间结构。我们在化学语言上对 Transformer 模型进行预训练，并朝向药物设计目标进行微调，研究高频 SMILES 子字符串与分子片段的对应关系。结果表明，语言模型可以从分子片段的角度理解化学结构，并且通过微调学到的结构知识反映在模型生成的高频 SMILES 子字符串中。