Biometric Verification (BV) systems often exhibit accuracy disparities across
different demographic groups, leading to biases in BV applications. Assessing
and quantifying these biases is essential for ensuring the fairness of BV
systems. However, existing bias evaluation metrics in BV have limitations, such
as focusing exclusively on match or non-match error rates, overlooking bias on
demographic groups with performance levels falling between the best and worst
performance levels, and neglecting the magnitude of the bias present.
This paper presents an in-depth analysis of the limitations of current bias
evaluation metrics in BV and, through experimental analysis, demonstrates their
contextual suitability, merits, and limitations. Additionally, it introduces a
novel general-purpose bias evaluation measure for BV, the ``Sum of Group Error
Differences (SEDG)''. Our experimental results on controlled synthetic datasets
demonstrate the effectiveness of demographic bias quantification when using
existing metrics and our own proposed measure. We discuss the applicability of
the bias evaluation metrics in a set of simulated demographic bias scenarios
and provide scenario-based metric recommendations. Our code is publicly
available under https://github.com/alaaobeid/SEDG.

生物特征验证（BV）系统往往在不同的人群中显示准确性差异，引发应用中的偏见。评估和量化这些偏见对确保 BV 系统的公正性至关重要。然而，现有的 BV 偏见评估指标存在一些限制，如仅关注匹配或非匹配错误率、忽视性能介于最好和最差性能水平之间人群的偏见以及忽视偏见的强度。本文对 BV 中现有偏见评估指标的限制进行了深入分析，并通过实验分析展示了它们的背景适应性、优点和限制。此外，还引入了一种新颖的通用偏见评估度量指标，即 “组错误差之和（SEDG）”。我们在控制的合成数据集上的实验结果表明，当使用现有指标和我们自己提出的度量时，能够有效量化人口统计偏见。我们讨论了偏见评估指标在一组模拟人口统计偏见场景中的适用性，并提供了基于场景的指标建议。我们的代码在 https://github.com/alaaobeid/SEDG 上公开共享。