Designing reward functions that guide generative molecular design (GMD)
algorithms to desirable areas of chemical space is of critical importance in
AI-driven drug discovery. Traditionally, this has been a manual and error-prone
task; the selection of appropriate computational methods to approximate
biological assays is challenging and the aggregation of computed values into a
single score even more so, leading to potential reliance on trial-and-error
approaches. We propose a novel approach for automated reward configuration that
relies solely on experimental data, mitigating the challenges of manual reward
adjustment on drug discovery projects. Our method achieves this by constructing
a ranking over experimental data based on Pareto dominance over the
multi-objective space, then training a neural network to approximate the reward
function such that rankings determined by the predicted reward correlate with
those determined by the Pareto dominance relation. We validate our method using
two case studies. In the first study we simulate Design-Make-Test-Analyse
(DMTA) cycles by alternating reward function updates and generative runs guided
by that function. We show that the learned function adapts over time to yield
compounds that score highly with respect to evaluation functions taken from the
literature. In the second study we apply our algorithm to historical data from
four real drug discovery projects. We show that our algorithm yields reward
functions that outperform the predictive accuracy of human-defined functions,
achieving an improvement of up to 0.4 in Spearman's correlation against a
ground truth evaluation function that encodes the target drug profile for that
project. Our method provides an efficient data-driven way to configure reward
functions for GMD, and serves as a strong baseline for future research into
transformative approaches for the automation of drug discovery.

设计奖励函数以引导生成分子设计算法到化学空间中的可取区域对于基于人工智能的药物发现至关重要。我们提出了一种基于实验数据的自动奖励配置方法，通过构建基于帕累托优势的实验数据排序，并训练神经网络以近似奖励函数，使得预测奖励相关的排序与基于帕累托优势的排序一致。我们验证了该方法的有效性，并展示其在药物发现中胜过人工定义的函数的预测准确性，与目标药物评价函数相比，Spearman 相关性提高了 0.4。该方法为生成分子设计的奖励函数配置提供了高效的数据驱动方式，并为自动化药物发现的未来研究提供了强有力的基准。