Auditing fairness of decision-makers is now in high demand. To respond to
this social demand, several fairness auditing tools have been developed. The
focus of this study is to raise an awareness of the risk of malicious
decision-makers who fake fairness by abusing the auditing tools and thereby
deceiving the social communities. The question is whether such a fraud of the
decision-maker is detectable so that the society can avoid the risk of fake
fairness. In this study, we answer this question negatively. We specifically
put our focus on a situation where the decision-maker publishes a benchmark
dataset as the evidence of his/her fairness and attempts to deceive a person
who uses an auditing tool that computes a fairness metric. To assess the
(un)detectability of the fraud, we explicitly construct an algorithm, the
stealthily biased sampling, that can deliberately construct an evil benchmark
dataset via subsampling. We show that the fraud made by the stealthily based
sampling is indeed difficult to detect both theoretically and empirically.

本研究关注于公平性决策者的欺诈风险并且提出了一种难以检测的算法 —— 隐秘偏倚抽样来满足这种欺诈需求。