Designing reward functions that guide generative molecular design (GMD)
algorithms to desirable areas of chemical space is of critical importance in
AI-driven drug discovery. Traditionally, this has been a manual and error-prone
task; the selection of appropriate computational methods to approximate
biological assays is challenging and the aggregation of computed values into a
single score even more so, leading to potential reliance on trial-and-error
approaches. We propose a novel approach for automated reward configuration that
relies solely on experimental data, mitigating the challenges of manual reward
adjustment on drug discovery projects. Our method achieves this by constructing
a ranking over experimental data based on Pareto dominance over the
multi-objective space, then training a neural network to approximate the reward
function such that rankings determined by the predicted reward correlate with
those determined by the Pareto dominance relation. We validate our method using
two case studies. In the first study we simulate Design-Make-Test-Analyse
(DMTA) cycles by alternating reward function updates and generative runs guided
by that function. We show that the learned function adapts over time to yield
compounds that score highly with respect to evaluation functions taken from the
literature. In the second study we apply our algorithm to historical data from
four real drug discovery projects. We show that our algorithm yields reward
functions that outperform the predictive accuracy of human-defined functions,
achieving an improvement of up to 0.4 in Spearman's correlation against a
ground truth evaluation function that encodes the target drug profile for that
project. Our method provides an efficient data-driven way to configure reward
functions for GMD, and serves as a strong baseline for future research into
transformative approaches for the automation of drug discovery.

设计奖励函数以引导生成分子设计算法到化学空间中的可取区域对于基于人工智能的药物发现至关重要。我们提出了一种基于实验数据的自动奖励配置方法，通过构建基于帕累托优势的实验数据排序，并训练神经网络以近似奖励函数，使得预测奖励相关的排序与基于帕累托优势的排序一致。我们验证了该方法的有效性，并展示其在药物发现中胜过人工定义的函数的预测准确性，与目标药物评价函数相比，Spearman 相关性提高了 0.4。该方法为生成分子设计的奖励函数配置提供了高效的数据驱动方式，并为自动化药物发现的未来研究提供了强有力的基准。

药物设计的奖励函数配置自动化

Automating reward function configuration for drug design

Generative molecular design has moved from proof-of-concept to real-world
applicability, as marked by the surge in very recent papers reporting
experimental validation. Key challenges in explainability and sample efficiency
present opportunities to enhance generative design to directly optimize
expensive high-fidelity oracles and provide actionable insights to domain
experts. Here, we propose Beam Enumeration to exhaustively enumerate the most
probable sub-sequences from language-based molecular generative models and show
that molecular substructures can be extracted. When coupled with reinforcement
learning, extracted substructures become meaningful, providing a source of
explainability and improving sample efficiency through self-conditioned
generation. Beam Enumeration is generally applicable to any language-based
molecular generative model and notably further improves the performance of the
recently reported Augmented Memory algorithm, which achieved the new
state-of-the-art on the Practical Molecular Optimization benchmark for sample
efficiency. The combined algorithm generates more high reward molecules and
faster, given a fixed oracle budget. Beam Enumeration is the first method to
jointly address explainability and sample efficiency for molecular design.

基于语言的分子生成模型的粒子截取方法可提取分子亚结构，通过强化学习使所提取的亚结构具有意义，以提供解释性，并通过自调整生成来提高样本效率。