Protein sequence design, determined by amino acid sequences, are essential to
protein engineering problems in drug discovery. Prior approaches have resorted
to evolutionary strategies or Monte-Carlo methods for protein design, but often
fail to exploit the structure of the combinatorial search space, to generalize
to unseen sequences. In the context of discrete black box optimization over
large search spaces, learning a mutation policy to generate novel sequences
with reinforcement learning is appealing. Recent advances in protein language
models (PLMs) trained on large corpora of protein sequences offer a potential
solution to this problem by scoring proteins according to their biological
plausibility (such as the TM-score). In this work, we propose to use PLMs as a
reward function to generate new sequences. Yet the PLM can be computationally
expensive to query due to its large size. To this end, we propose an
alternative paradigm where optimization can be performed on scores from a
smaller proxy model that is periodically finetuned, jointly while learning the
mutation policy. We perform extensive experiments on various sequence lengths
to benchmark RL-based approaches, and provide comprehensive evaluations along
biological plausibility and diversity of the protein. Our experimental results
include favorable evaluations of the proposed sequences, along with high
diversity scores, demonstrating that RL is a strong candidate for biological
sequence design. Finally, we provide a modular open source implementation can
be easily integrated in most RL training loops, with support for replacing the
reward model with other PLMs, to spur further research in this domain. The code
for all experiments is provided in the supplementary material.

蛋白质序列设计对于药物研发中的蛋白质工程问题至关重要，本研究提出使用蛋白质语言模型作为奖励函数来生成新的序列，利用强化学习和最优化方法进行体系建模，实验证明强化学习在生物序列设计方面具有很大潜力。

利用蛋白质语言模型进行序列设计的强化学习

Reinforcement Learning for Sequence Design Leveraging Protein Language  Models

Bayesian optimization (BayesOpt) is a gold standard for query-efficient
continuous optimization. However, its adoption for drug design has been
hindered by the discrete, high-dimensional nature of the decision variables. We
develop a new approach (LaMBO) which jointly trains a denoising autoencoder
with a discriminative multi-task Gaussian process head, allowing gradient-based
optimization of multi-objective acquisition functions in the latent space of
the autoencoder. These acquisition functions allow LaMBO to balance the
explore-exploit tradeoff over multiple design rounds, and to balance objective
tradeoffs by optimizing sequences at many different points on the Pareto
frontier. We evaluate LaMBO on two small-molecule design tasks, and introduce
new tasks optimizing \emph{in silico} and \emph{in vitro} properties of
large-molecule fluorescent proteins. In our experiments LaMBO outperforms
genetic optimizers and does not require a large pretraining corpus,
demonstrating that BayesOpt is practical and effective for biological sequence
design.

LaMBO 是一种新的 Bayesian optimization 方法，采用了联合训练的方法，结合 Denoising Autoencoder 和多任务 Gaussian Process Head 进行多目标优化，以实现在离散高维空间中进行生物序列设计。

使用去噪自编码器加速生物序列设计的贝叶斯优化

Accelerating Bayesian Optimization for Biological Sequence Design with  Denoising Autoencoders

Black-box optimization formulations for biological sequence design have drawn
recent attention due to their promising potential impact on the pharmaceutical
industry. In this work, we propose to unify two seemingly distinct worlds:
likelihood-free inference and black-box optimization, under one probabilistic
framework. In tandem, we provide a recipe for constructing various sequence
design methods based on this framework. We show how previous optimization
approaches can be "reinvented" in our framework, and further propose new
probabilistic black-box optimization algorithms. Extensive experiments on
sequence design application illustrate the benefits of the proposed
methodology.

本文提出将无似然推断和黑盒优化统一到一个概率框架下的构造方法，以此为基础构建序列设计方法，并提出新的概率黑盒优化算法，通过序列设计应用的广泛实验表明该方法的优势。