Interpretability and fairness are critical in computer vision and machine
learning applications, in particular when dealing with human outcomes, e.g.
inviting or not inviting for a job interview based on application materials
that may include photographs. One promising direction to achieve fairness is by
learning data representations that remove the semantics of protected
characteristics, and are therefore able to mitigate unfair outcomes. All
available models however learn latent embeddings which comes at the cost of
being uninterpretable. We propose to cast this problem as data-to-data
translation, i.e. learning a mapping from an input domain to a fair target
domain, where a fairness definition is being enforced. Here the data domain can
be images, or any tabular data representation. This task would be
straightforward if we had fair target data available, but this is not the case.
To overcome this, we learn a highly unconstrained mapping by exploiting
statistics of residuals - the difference between input data and its translated
version - and the protected characteristics. When applied to the CelebA dataset
of face images with gender attribute as the protected characteristic, our model
enforces equality of opportunity by adjusting the eyes and lips regions.
Intriguingly, on the same dataset we arrive at similar conclusions when using
semantic attribute representations of images for translation. On face images of
the recent DiF dataset, with the same gender attribute, our method adjusts nose
regions. In the Adult income dataset, also with protected gender attribute, our
model achieves equality of opportunity by, among others, obfuscating the wife
and husband relationship. Analyzing those systematic changes will allow us to
scrutinize the interplay of fairness criterion, chosen protected
characteristics, and prediction performance.

本文提出了一种数据之间的翻译方法，实现了图像或表格数据的公平性，有效避免保护特征的语义性对公平性造成的影响，并在实际数据集面对性别这一保护特征时表现出了很好的机会平等性。