Artificial Intelligence (AI) has significantly revolutionized radiology,
promising improved patient outcomes and streamlined processes. However, it's
critical to ensure the fairness of AI models to prevent stealthy bias and
disparities from leading to unequal outcomes. This review discusses the concept
of fairness in AI, focusing on bias auditing using the Aequitas toolkit, and
its real-world implications in radiology, particularly in disease screening
scenarios. Aequitas, an open-source bias audit toolkit, scrutinizes AI models'
decisions, identifying hidden biases that may result in disparities across
different demographic groups and imaging equipment brands. This toolkit
operates on statistical theories, analyzing a large dataset to reveal a model's
fairness. It excels in its versatility to handle various variables
simultaneously, especially in a field as diverse as radiology. The review
explicates essential fairness metrics: Equal and Proportional Parity, False
Positive Rate Parity, False Discovery Rate Parity, False Negative Rate Parity,
and False Omission Rate Parity. Each metric serves unique purposes and offers
different insights. We present hypothetical scenarios to demonstrate their
relevance in disease screening settings, and how disparities can lead to
significant real-world impacts.

本文评述了人工智能在放射学中的应用，特别关注了平等和比例性公正等算法的实际使用以及如何使用开源的 Aequitas 工具包进行偏见审计，以确保数据集的公正性和避免 AI 模型中的隐含偏见。

放射学人工智能中的公平性导航：概念、后果和关键考虑

Navigating Fairness in Radiology AI: Concepts, Consequences,and Crucial  Considerations

Auditing unwanted social bias in language models (LMs) is inherently hard due
to the multidisciplinary nature of the work. In addition, the rapid evolution
of LMs can make benchmarks irrelevant in no time. Bias auditing is further
complicated by LM brittleness: when a presumably biased outcome is observed, is
it due to model bias or model brittleness? We propose enlisting the models
themselves to help construct bias auditing datasets that remain challenging,
and introduce bias measures that distinguish between types of model errors.
First, we extend an existing bias benchmark for NLI (BBNLI) using a combination
of LM-generated lexical variations, adversarial filtering, and human
validation. We demonstrate that the newly created dataset (BBNLInext) is more
challenging than BBNLI: on average, BBNLI-next reduces the accuracy of
state-of-the-art NLI models from 95.3%, as observed by BBNLI, to 58.6%. Second,
we employ BBNLI-next to showcase the interplay between robustness and bias, and
the subtlety in differentiating between the two. Third, we point out
shortcomings in current bias scores used in the literature and propose bias
measures that take into account pro-/anti-stereotype bias and model
brittleness. We will publicly release the BBNLI-next dataset to inspire
research on rapidly expanding benchmarks to keep up with model evolution, along
with research on the robustness-bias interplay in bias auditing.
Note: This paper contains offensive text examples.

本文提出了一种利用语言模型生成词汇变体、对抗性过滤和人工验证相结合的方法扩展现有的 Bias Benchmark for NLI (BBNLI) 测评数据集为 BBNLI-next，并介绍了能够区分模型误差类型的倾向度（bias）测量标准，说明了现有偏见分数存在的不足并提出了考虑到促进 / 反对刻板印象倾向和模型脆弱性的倾向度测量标准。