Recently, transductive learning methods, which leverage holdout sets during
training, have gained popularity for their potential to improve speed,
accuracy, and fairness in machine learning models. Despite this, the
composition of the holdout set itself, particularly the balance of sensitive
sub-groups, has been largely overlooked. Our experiments on CIFAR and CelebA
datasets show that compositional changes in the holdout set can substantially
influence fairness metrics. Imbalanced holdout sets exacerbate existing
disparities, while balanced holdouts can mitigate issues introduced by
imbalanced training data. These findings underline the necessity of
constructing holdout sets that are both diverse and representative.

最近，在机器学习模型的速度、准确性和公平性方面改进的潜力上，利用留置集在训练过程中的转导学习方法已经越来越受欢迎。然而，留置集本身的构成，特别是敏感子群的平衡，却一直被忽视。我们在 CIFAR 和 CelebA 数据集上的实验表明，留置集中的构成变化可以大大影响公平度量。失衡的留置集加剧了现有的不平等问题，而平衡的留置集可以缓解由于不平衡训练数据引入的问题。这些发现强调了构建既多样又具有代表性的留置集的必要性。