Medical machine learning algorithms are typically evaluated based on accuracy
vs. a clinician-defined ground truth, a reasonable choice because trained
clinicians are usually better classifiers than ML models. However, this metric
does not fully reflect the clinical task: it neglects the fact that humans,
even with perfect accuracy, are subject to sometimes significant error from the
Poisson statistics of rare events, because clinical protocols often specify
that a relatively small sample be examined. For example, to quantitate malaria
on a thin blood film a clinician examines only 2000 red blood cells (0.0004
uL), which can yield large variation in actual number of parasites present due
to Poisson variability, so that a perfect human's count can differ
substantially from the true average load. In contrast, ML systems may be less
accurate on an object level, but they also may have the option to examine more
blood (e.g. 0.1 uL, or 250x). So while their accuracy as to parasite count in a
particular sample is lower, the Poisson variability of their estimate is also
lower due to larger sample size.
Crucially, when an ML system moves out of the proof-of-concept stage and
targets deployment in a clinical setting, its performance must match current
standard of care. To this end, it may have the option to offset its lower
accuracy by increasing sample size to reduce Poisson error, and thus attain the
same net clinical performance as a perfectly accurate human limited by smaller
sample size. In this paper, we analyze the mathematics of the trade-off between
these two types of error, to enable teams developing ML systems to leverage a
relative strength (larger sample sizes) to offset a relative weakness
(classification accuracy). We illustrate the methods with two concrete
examples: diagnosis and quantitation of malaria on blood films.

在医疗机器学习算法中，通过比较与临床医生定义的标准结果的准确性来评估该算法往往是合理的选择。然而，这并不能完全反映临床任务的实际情况：它忽视了人类由于罕见事件的 Poisson 统计学规律而产生的有时相当大的误差以及临床协议通常要求只检查相对较小样本的事实。相比之下，机器学习系统可以通过检查更多的血液样本来弥补其准确性上的不足，因此达到与仅限于较小样本的准确人工判读相同的实际临床表现。本文分析了这两种错误之间的权衡数学关系，以帮助开发机器学习系统的团队利用相对优势（更大的样本量）来弥补相对劣势（分类准确性），并以疟疾的诊断和定量为例说明了这些方法。

降低泊松误差可以抵消临床任务中的分类误差

Driving down Poisson error can offset classification error in clinical  tasks

DNA-encoded library (DEL) screening and quantitative structure-activity
relationship (QSAR) modeling are two techniques used in drug discovery to find
small molecules that bind a protein target. Applying QSAR modeling to DEL data
can facilitate the selection of compounds for off-DNA synthesis and evaluation.
Such a combined approach has been shown recently by training binary classifiers
to learn DEL enrichments of aggregated "disynthons" to accommodate the sparse
and noisy nature of DEL data. However, a binary classifier cannot distinguish
between different levels of enrichment, and information is potentially lost
during disynthon aggregation. Here, we demonstrate a regression approach to
learning DEL enrichments of individual molecules using a custom negative
log-likelihood loss function that effectively denoises DEL data and introduces
opportunities for visualization of learned structure-activity relationships
(SAR). Our approach explicitly models the Poisson statistics of the sequencing
process used in the DEL experimental workflow under a frequentist view. We
illustrate this approach on a dataset of 108k compounds screened against CAIX,
and a dataset of 5.7M compounds screened against sEH and SIRT2. Due to the
treatment of uncertainty in the data through the negative log-likelihood loss
function, the models can ignore low-confidence outliers. While our approach
does not demonstrate a benefit for extrapolation to novel structures, we expect
our denoising and visualization pipeline to be useful in identifying SAR trends
and enriched pharmacophores in DEL data. Further, this approach to
uncertainty-aware regression is applicable to other sparse or noisy datasets
where the nature of stochasticity is known or can be modeled; in particular,
the Poisson enrichment ratio metric we use can apply to other settings that
compare sequencing count data between two experimental conditions.

本研究中使用回归方法和 Poisson 统计学建模 DEL 数据，计算药物的定量结构 - 活性关系并可视化学习的结果。