Recent breakthroughs by deep neural networks (DNNs) in natural language
processing (NLP) and computer vision have been driven by a scale-up of models
and data rather than the discovery of novel computing paradigms. Here, we
investigate if scale can have a similar impact for models designed to aid small
molecule drug discovery. We address this question through a large-scale and
systematic analysis of how DNN size, data diet, and learning routines interact
to impact accuracy on our Phenotypic Chemistry Arena (Pheno-CA) benchmark: a
diverse set of drug development tasks posed on image-based high content
screening data. Surprisingly, we find that DNNs explicitly supervised to solve
tasks in the Pheno-CA do not continuously improve as their data and model size
is scaled-up. To address this issue, we introduce a novel precursor task, the
Inverse Biological Process (IBP), which is designed to resemble the causal
objective functions that have proven successful for NLP. We indeed find that
DNNs first trained with IBP then probed for performance on the Pheno-CA
significantly outperform task-supervised DNNs. More importantly, the
performance of these IBP-trained DNNs monotonically improves with data and
model scale. Our findings reveal that the DNN ingredients needed to accurately
solve small molecule drug development tasks are already in our hands, and
project how much more experimental data is needed to achieve any desired level
of improvement. We release our Pheno-CA benchmark and code to encourage further
study of neural scaling laws for small molecule drug discovery.

我们发现，深度神经网络在小分子药物研究任务中的精确解决所需的关键因素已经掌握，同时通过引入一种新的前体任务，即逆生物过程 (IBP)，可以进一步提高性能并实现更多实验数据所需的改进级别。

药物发现的神经缩放法则

Neural scaling laws for phenotypic drug discovery

We address the task of controlled generation of small molecules, which
entails finding novel molecules with desired properties under certain
constraints (e.g., similarity to a reference molecule). Here we introduce
MolMIM, a probabilistic auto-encoder for small molecule drug discovery that
learns an informative and clustered latent space. MolMIM is trained with Mutual
Information Machine (MIM) learning, and provides a fixed length representation
of variable length SMILES strings. Since encoder-decoder models can learn
representations with ``holes'' of invalid samples, here we propose a novel
extension to the training procedure which promotes a dense latent space, and
allows the model to sample valid molecules from random perturbations of latent
codes. We provide a thorough comparison of MolMIM to several variable-size and
fixed-size encoder-decoder models, demonstrating MolMIM's superior generation
as measured in terms of validity, uniqueness, and novelty. We then utilize
CMA-ES, a naive black-box and gradient free search algorithm, over MolMIM's
latent space for the task of property guided molecule optimization. We achieve
state-of-the-art results in several constrained single property optimization
tasks as well as in the challenging task of multi-objective optimization,
improving over previous success rate SOTA by more than 5\% . We attribute the
strong results to MolMIM's latent representation which clusters similar
molecules in the latent space, whereas CMA-ES is often used as a baseline
optimization method. We also demonstrate MolMIM to be favourable in a compute
limited regime, making it an attractive model for such cases.

该论文提出一种基于自编码器和互信息机器学习的方法 MolMIM，用于小分子药物发现领域的分子生成和优化，并使用 CMA-ES 算法在 MolMIM 潜在空间中进行性质引导的分子优化任务，取得了优异的结果。