Language modeling has seen impressive progress over the last years, mainly
prompted by the invention of the Transformer architecture, sparking a
revolution in many fields of machine learning, with breakthroughs in chemistry
and biology. In this chapter, we explore how analogies between chemical and
natural language have inspired the use of Transformers to tackle important
bottlenecks in the drug discovery process, such as retrosynthetic planning and
chemical space exploration. The revolution started with models able to perform
particular tasks with a single type of data, like linearised molecular graphs,
which then evolved to include other types of data, like spectra from analytical
instruments, synthesis actions, and human language. A new trend leverages
recent developments in large language models, giving rise to a wave of models
capable of solving generic tasks in chemistry, all facilitated by the
flexibility of natural language. As we continue to explore and harness these
capabilities, we can look forward to a future where machine learning plays an
even more integral role in accelerating scientific discovery.

利用自然语言与化学的类比，本文探索了利用 Transformers 处理药物发现过程中的重要障碍，如后合成规划和化学空间探索，并展示了大型语言模型在化学领域解决通用任务的潜力。

化学与药物发现中的 Transformer 和大型语言模型

Transformers and Large Language Models for Chemistry and Drug Discovery

Multiple sclerosis (MS) is a debilitating neurological disease affecting
nearly one million people in the United States. Sphingosine-1-phosphate
receptor 1, or S1PR1, is a protein target for MS. Siponimod, a ligand of S1PR1,
was approved by the FDA in 2019 for MS treatment, but there is a demonstrated
need for better therapies. To this end, we finetuned an autoencoder machine
learning model that converts chemical formulas into mathematical vectors and
generated over 500 molecular variants based on siponimod, out of which 25
compounds had higher predicted binding affinity to S1PR1. The model was able to
generate these ligands in just under one hour. Filtering these compounds led to
the discovery of six promising candidates with good drug-like properties and
ease of synthesis. Furthermore, by analyzing the binding interactions for these
ligands, we uncovered several chemical properties that contribute to high
binding affinity to S1PR1. This study demonstrates that machine learning can
accelerate the drug discovery process and reveal new insights into protein-drug
interactions.

通过细化自动编码器机器学习模型，将化学配方转化为数学向量，并生成超过 500 种基于 siponimod 的分子变体，其中 25 种化合物对 S1PR1 具有更高的预测结合亲和力；筛选这些化合物后，发现了 6 个具有良好药物特性和易合成功能的有希望的候选物，还通过分析这些配体的结合相互作用，揭示了几种对 S1PR1 高结合亲和力的化学性质。本研究证明了机器学习可以加速药物发现过程，揭示蛋白质 - 药物相互作用的新见解。