Deep learning models have shown great potential for image-based diagnosis
assisting clinical decision making. At the same time, an increasing number of
reports raise concerns about the potential risk that machine learning could
amplify existing health disparities due to human biases that are embedded in
the training data. It is of great importance to carefully investigate the
extent to which biases may be reproduced or even amplified if we wish to build
fair artificial intelligence systems. Seyyed-Kalantari et al. advance this
conversation by analysing the performance of a disease classifier across
population subgroups. They raise performance disparities related to
underdiagnosis as a point of concern; we identify areas from this analysis
which we believe deserve additional attention. Specifically, we wish to
highlight some theoretical and practical difficulties associated with assessing
model fairness through testing on data drawn from the same biased distribution
as the training data, especially when the sources and amount of biases are
unknown.

该研究通过对人群亚组之间的疾病分类器性能分析，发现低诊断率造成的表现差异引起了关注，进一步强调了在训练数据中嵌入的人类偏见可能会放大现有的健康差异，并探讨了评估模型公正性的一些困难。