In machine learning (ML), a widespread adage is that the area under the
precision-recall curve (AUPRC) is a superior metric for model comparison to the
area under the receiver operating characteristic (AUROC) for binary
classification tasks with class imbalance. This paper challenges this notion
through novel mathematical analysis, illustrating that AUROC and AUPRC can be
concisely related in probabilistic terms. We demonstrate that AUPRC, contrary
to popular belief, is not superior in cases of class imbalance and might even
be a harmful metric, given its inclination to unduly favor model improvements
in subpopulations with more frequent positive labels. This bias can
inadvertently heighten algorithmic disparities. Prompted by these insights, a
thorough review of existing ML literature was conducted, utilizing large
language models to analyze over 1.5 million papers from arXiv. Our
investigation focused on the prevalence and substantiation of the purported
AUPRC superiority. The results expose a significant deficit in empirical
backing and a trend of misattributions that have fuelled the widespread
acceptance of AUPRC's supposed advantages. Our findings represent a dual
contribution: a significant technical advancement in understanding metric
behaviors and a stark warning about unchecked assumptions in the ML community.
All experiments are accessible at
this https URL

通过新颖的数学分析，本文挑战并证明按照概率的观点，AUPRC（精确率 - 召回率曲线下的面积）和 AUROC（接收者操作特征曲线下的面积）可以简洁地相互关联。同时，通过对超过 150 万篇论文的大规模语言模型分析，揭示了 AUPRC 被广泛接受的先入为主观念的实证不足和误导趋势，强调了对度量标准性能的深入理解的重要性和机器学习社区中未经证实的假设的危险性。