In recent years, the number of new applications for highly complex AI systems
has risen significantly. Algorithmic decision-making systems (ADMs) are one of
such applications, where an AI system replaces the decision-making process of a
human expert. As one approach to ensure fairness and transparency of such
systems, explainable AI (XAI) has become more important. One variant to achieve
explainability are surrogate models, i.e., the idea to train a new simpler
machine learning model based on the input-output-relationship of a black box
model. The simpler machine learning model could, for example, be a decision
tree, which is thought to be intuitively understandable by humans. However,
there is not much insight into how well the surrogate model approximates the
black box.
Our main assumption is that a good surrogate model approach should be able to
bring such a discriminating behavior to the attention of humans; prior to our
research we assumed that a surrogate decision tree would identify such a
pattern on one of its first levels. However, in this article we show that even
if the discriminated subgroup - while otherwise being the same in all
categories - does not get a single positive decision from the black box ADM
system, the corresponding question of group membership can be pushed down onto
a level as low as wanted by the operator of the system.
We then generalize this finding to pinpoint the exact level of the tree on
which the discriminating question is asked and show that in a more realistic
scenario, where discrimination only occurs to some fraction of the
disadvantaged group, it is even more feasible to hide such discrimination.
Our approach can be generalized easily to other surrogate models.

最近几年，高度复杂人工智能系统的新应用数量大幅增长。算法决策系统 (ADMs) 是其中之一，其通过人工智能系统代替人类专家的决策过程。为确保此类系统的公平性和透明性，可解释人工智能 (XAI) 越来越重要。一种实现可解释性的变体是 surrogate 模型，即基于黑盒模型的输入 - 输出关系训练新的简化的机器学习模型的思想。简化的机器学习模型可以是决策树，被认为对人类直观可理解。然而，关于 surrogate 模型多好地逼近黑盒模型，我们知之甚少。本文的主要假设是一个良好的 surrogate 模型方法应能引起人类的注意力，先前我们假定一个 surrogate 决策树在其前几层将识别出这种歧视模式。然而，在本文中我们展示，即使被歧视的子群体在所有类别上与其他相同，也不会从黑盒 ADM 系统获得任何正面决策，相应的群体成员问题可以由系统操作者推迟到任意低的层级。我们进一步将此发现推广到准确定位歧视问题被问及的决策树层级，并展示在更现实的场景中，即只有某一部分处于劣势的群体才发生歧视，隐藏这种歧视行为更加可行。我们的方法可以轻易应用于其他 surrogate 模型。

利用替代模型方法进行 XAI 的黑客攻击

Hacking a surrogate model approach to XAI

As algorithmic decision-making systems become more prevalent in society,
ensuring the fairness of these systems is becoming increasingly important.
Whilst there has been substantial research in building fair algorithmic
decision-making systems, the majority of these methods require access to the
training data, including personal characteristics, and are not transparent
regarding which individuals are classified unfairly. In this paper, we propose
a novel model-agnostic argumentation-based method to determine why an
individual is classified differently in comparison to similar individuals. Our
method uses a quantitative argumentation framework to represent attribute-value
pairs of an individual and of those similar to them, and uses a well-known
semantics to identify the attribute-value pairs in the individual contributing
most to their different classification. We evaluate our method on two datasets
commonly used in the fairness literature and illustrate its effectiveness in
the identification of bias.

通过使用一种新颖的模型不可知论证方法，本论文研究了为何一个个体的分类与其他相似个体不同，设计了一个定量论证框架来表示个体及其相似个体的属性 - 值对，并利用已知语义来识别对个体不同分类起到最大贡献的属性 - 值对，本方法在两个常用的公平性数据集上进行了评估，证明了其鉴定偏见的有效性。

识别偏见的原因：基于论证的方法

Identifying Reasons for Bias: An Argumentation-Based Approach

Artificial Intelligence (AI) is rapidly integrating into various aspects of
our daily lives, influencing decision-making processes in areas such as
targeted advertising and matchmaking algorithms. As AI systems become
increasingly sophisticated, ensuring their transparency and explainability
becomes crucial. Functional transparency is a fundamental aspect of algorithmic
decision-making systems, allowing stakeholders to comprehend the inner workings
of these systems and enabling them to evaluate their fairness and accuracy.
However, achieving functional transparency poses significant challenges that
need to be addressed. In this paper, we propose a design for user-centered
compliant-by-design transparency in transparent systems. We emphasize that the
development of transparent and explainable AI systems is a complex and
multidisciplinary endeavor, necessitating collaboration among researchers from
diverse fields such as computer science, artificial intelligence, ethics, law,
and social science. By providing a comprehensive understanding of the
challenges associated with transparency in AI systems and proposing a
user-centered design framework, we aim to facilitate the development of AI
systems that are accountable, trustworthy, and aligned with societal values.

人工智能的功能透明性在透明系统中遵守设计框架中成为关键挑战，要求跨学科合作来推进透明和可解释的人工智能系统的发展。