Real-world datasets are typically imbalanced in the sense that only a few
classes have numerous samples, while many classes are associated with only a
few samples. As a result, a na\"ive ERM learning process will be biased towards
the majority classes, making it difficult to generalize to the minority
classes. To address this issue, one simple but effective approach is to modify
the loss function to emphasize the learning on minority classes, such as
re-weighting the losses or adjusting the logits via class-dependent terms.
However, existing generalization analysis of such losses is still
coarse-grained and fragmented, failing to explain some empirical results. To
bridge this gap, we propose a novel technique named data-dependent contraction
to capture how these modified losses handle different classes. On top of this
technique, a fine-grained generalization bound is established for imbalanced
learning, which helps reveal the mystery of re-weighting and logit-adjustment
in a unified manner. Furthermore, a principled learning algorithm is developed
based on the theoretical insights. Finally, the empirical results on benchmark
datasets not only validate the theoretical results but also demonstrate the
effectiveness of the proposed method.

通过对失衡学习的概括分析和数据依赖性收缩等技术，建立了一个精细的不平衡学习泛化界限，进一步揭示了重新加权和逻辑调整的原理，并开发了一个基于理论洞察的原则学习算法。

不平衡学习中加权重和对数调整的统一概化分析

A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment  for Imbalanced Learning

Gender bias exists in natural language datasets which neural language models
tend to learn, resulting in biased text generation. In this research, we
propose a debiasing approach based on the loss function modification. We
introduce a new term to the loss function which attempts to equalize the
probabilities of male and female words in the output. Using an array of bias
evaluation metrics, we provide empirical evidence that our approach
successfully mitigates gender bias in language models without increasing
perplexity. In comparison to existing debiasing strategies, data augmentation,
and word embedding debiasing, our method performs better in several aspects,
especially in reducing gender bias in occupation words. Finally, we introduce a
combination of data augmentation and our approach, and show that it outperforms
existing strategies in all bias evaluation metrics.

本研究提出一种基于损失函数修改的去偏见方法，旨在平衡输出中男性和女性单词的概率；通过使用一系列偏差评估指标，我们提供实证证据表明我们的方法成功地减轻了语言模型中的性别偏见。与现有的去偏见策略，数据扩充和单词嵌入去偏见相比，我们的方法在多个方面，特别是在减少职业单词中的性别偏见方面表现更好。最后，我们介绍了一种数据扩充和我们方法的组合，并展示它在所有偏差评估指标中表现出优于现有策略的效果。