Recent dataset deduplication techniques have demonstrated that content-aware
dataset pruning can dramatically reduce the cost of training Vision-Language
Pretrained (VLP) models without significant performance losses compared to
training on the original dataset. These results have been based on pruning
commonly used image-caption datasets collected from the web -- datasets that
are known to harbor harmful social biases that may then be codified in trained
models. In this work, we evaluate how deduplication affects the prevalence of
these biases in the resulting trained models and introduce an easy-to-implement
modification to the recent SemDeDup algorithm that can reduce the negative
effects that we observe. When examining CLIP-style models trained on
deduplicated variants of LAION-400M, we find our proposed FairDeDup algorithm
consistently leads to improved fairness metrics over SemDeDup on the FairFace
and FACET datasets while maintaining zero-shot performance on CLIP benchmarks.

通过数据集去重技术，以及针对包含社会偏见的图像 - 描述数据集的剪枝技术，本研究评估去重对训练模型中偏见的影响，并提出了一种易于实施的修改后的 SemDeDup 算法，以减少这些负面影响，实验证明我们提出的 FairDeDup 算法在 FairFace 和 FACET 数据集上提供了更好的公平性指标，并保持了 CLIP benchmark 的零 - shot 性能。