Neural networks trained with class-imbalanced data are known to perform
poorly on minor classes of scarce training data. Several recent works attribute
this to over-fitting to minor classes. In this paper, we provide a novel
explanation of this issue. We found that a neural network tends to first
under-fit the minor classes by classifying most of their data into the major
classes in early training epochs. To correct these wrong predictions, the
neural network then must focus on pushing features of minor class data across
the decision boundaries between major and minor classes, leading to much larger
gradients for features of minor classes. We argue that such an under-fitting
phase over-emphasizes the competition between major and minor classes, hinders
the neural network from learning the discriminative knowledge that can be
generalized to test data, and eventually results in over-fitting. To address
this issue, we propose a novel learning strategy to equalize the training
progress across classes. We mix features of the major class data with those of
other data in a mini-batch, intentionally weakening their features to prevent a
neural network from fitting them first. We show that this strategy can largely
balance the training accuracy and feature gradients across classes, effectively
mitigating the under-fitting then over-fitting problem for minor class data. On
several benchmark datasets, our approach achieves the state-of-the-art
accuracy, especially for the challenging step-imbalanced cases.

本文提出了一个新的学习策略，来有效地解决在使用类不平衡数据时神经网络容易出现欠拟合和过拟合问题的困扰，该策略在小批量中混合了主类数据和其它数据的特征，旨在削弱主类数据的特征以防止神经网络优先拟合主类数据的情况，从而实现了类之间的训练过程平衡，并得到了在难度较大的数据集中的最新成果。