We show that established performance metrics in binary classification, such
as the F-score, the Jaccard similarity coefficient or Matthews' correlation
coefficient (MCC), are not robust to class imbalance in the sense that if the
proportion of the minority class tends to $0$, the true positive rate (TPR) of
the Bayes classifier under these metrics tends to $0$ as well. Thus, in
imbalanced classification problems, these metrics favour classifiers which
ignore the minority class. To alleviate this issue we introduce robust
modifications of the F-score and the MCC for which, even in strongly imbalanced
settings, the TPR is bounded away from $0$. We numerically illustrate the
behaviour of the various performance metrics in simulations as well as on a
credit default data set. We also discuss connections to the ROC and
precision-recall curves and give recommendations on how to combine their usage
with performance metrics.

在不平衡分类问题中，传统的性能度量标准如 F-score、Jaccard 相似系数或 Matthews 相关系数对于类别不平衡并不稳健，因为当少数类占比接近 0 时，这些度量标准下的贝叶斯分类器的真正阳性率（TPR）也趋近于 0。为了解决这个问题，我们提出了对 F-score 和 MCC 的稳健修正，即使在极度不平衡的情况下，TPR 也得到了保证。我们通过模拟和信用违约数据集的实验来展示各种性能度量标准的行为，并讨论了与 ROC 和精确率 - 召回率曲线的关联性，并提出了如何将它们与性能度量标准结合使用的建议。