With the emergence of Transformer architectures and their powerful
understanding of textual data, a new horizon has opened up to predict the
molecular properties based on text description. While SMILES are the most
common form of representation, they are lacking robustness, rich information
and canonicity, which limit their effectiveness in becoming generalizable
representations. Here, we present GPT-MolBERTa, a self-supervised large
language model (LLM) which uses detailed textual descriptions of molecules to
predict their properties. A text based description of 326000 molecules were
collected using ChatGPT and used to train LLM to learn the representation of
molecules. To predict the properties for the downstream tasks, both BERT and
RoBERTa models were used in the finetuning stage. Experiments show that
GPT-MolBERTa performs well on various molecule property benchmarks, and
approaching state of the art performance in regression tasks. Additionally,
further analysis of the attention mechanisms show that GPT-MolBERTa is able to
pick up important information from the input textual data, displaying the
interpretability of the model.

利用自然语言处理模型 GPT-MolBERTa，通过分析分子的详细文本描述来预测其性质，并展示模型的可解释性。

GPT-MolBERTa: 用于分子性质预测的 GPT 分子特征语言模型

GPT-MolBERTa: GPT Molecular Features Language Model for molecular  property prediction

We present a machine learning algorithm for the prediction of molecule
properties inspired by ideas from density functional theory. Using
Gaussian-type orbital functions, we create surrogate electronic densities of
the molecule from which we compute invariant "solid harmonic scattering
coefficients" that account for different types of interactions at different
scales. Multi-linear regressions of various physical properties of molecules
are computed from these invariant coefficients. Numerical experiments show that
these regressions have near state of the art performance, even with relatively
few training examples. Predictions over small sets of scattering coefficients
can reach a DFT precision while being interpretable.

本文提出了一种基于密度泛函理论的分子性质预测机器学习算法，利用高斯型轨道函数创建分子的代理电子密度，并计算不变的 “固体谐波散射系数”，由此对不同尺度下的不同类型相互作用进行建模，并通过多线性回归计算出分子的各种物理性质，此算法在小型数据集方面已达到接近 DFT 预测精度，同时非常可解释。