Protein language models learn powerful representations directly from
sequences of amino acids. However, they are constrained to generate proteins
with only the set of amino acids represented in their vocabulary. In contrast,
chemical language models learn atom-level representations of smaller molecules
that include every atom, bond, and ring. In this work, we show that chemical
language models can learn atom-level representations of proteins enabling
protein generation unconstrained to the standard genetic code and far beyond
it. In doing so, we show that language models can generate entire proteins atom
by atom -- effectively learning the multiple hierarchical layers of molecular
information that define proteins from their primary sequence to their
secondary, and tertiary structure. We demonstrate language models are able to
explore beyond protein space -- generating proteins with modified sidechains
that form unnatural amino acids. Even further, we find that language models can
explore chemical space and protein space simultaneously and generate novel
examples of protein-drug conjugates. The results demonstrate the potential for
biomolecular design at the atom level using language models.

蛋白质语言模型和化学语言模型可以学习蛋白质的原子级表示，实现无限制的蛋白质生成，包括修改侧链和生成蛋白质 - 药物结合物，展示了语言模型在原子级别的生物分子设计的潜力。

逐个原子生成蛋白质及其超越：基于语言模型的研究

Atom-by-atom protein generation and beyond with language models

Explainability techniques are crucial in gaining insights into the reasons
behind the predictions of deep learning models, which have not yet been applied
to chemical language models. We propose an explainable AI technique that
attributes the importance of individual atoms towards the predictions made by
these models. Our method backpropagates the relevance information towards the
chemical input string and visualizes the importance of individual atoms. We
focus on self-attention Transformers operating on molecular string
representations and leverage a pretrained encoder for finetuning. We showcase
the method by predicting and visualizing solubility in water and organic
solvents. We achieve competitive model performance while obtaining
interpretable predictions, which we use to inspect the pretrained model.

该研究提出了一种可解释的 AI 技术来解释和预测化学模型，能够精确推断分子溶度和物理属性。