Corrupted labels and class imbalance are commonly encountered in practically
collected training data, which easily leads to over-fitting of deep neural
networks (DNNs). Existing approaches alleviate these issues by adopting a
sample re-weighting strategy, which is to re-weight sample by designing
weighting function. However, it is only applicable for training data containing
only either one type of data biases. In practice, however, biased samples with
corrupted labels and of tailed classes commonly co-exist in training data. How
to handle them simultaneously is a key but under-explored problem. In this
paper, we find that these two types of biased samples, though have similar
transient loss, have distinguishable trend and characteristics in loss curves,
which could provide valuable priors for sample weight assignment. Motivated by
this, we delve into the loss curves and propose a novel probe-and-allocate
training strategy: In the probing stage, we train the network on the whole
biased training data without intervention, and record the loss curve of each
sample as an additional attribute; In the allocating stage, we feed the
resulting attribute to a newly designed curve-perception network, named
CurveNet, to learn to identify the bias type of each sample and assign proper
weights through meta-learning adaptively. The training speed of meta learning
also blocks its application. To solve it, we propose a method named skip layer
meta optimization (SLMO) to accelerate training speed by skipping the bottom
layers. Extensive synthetic and real experiments well validate the proposed
method, which achieves state-of-the-art performance on multiple challenging
benchmarks.

该论文提出了一种名为 CurveNet 的曲线感知网络，并采用探测和分配的方法，通过元学习自适应地分配适当的样本权重，以解决深度神经网络中常见的数据偏见和样本不平衡问题，并利用 skip layer meta optimization 方法加速了训练速度，该方法在多项具有挑战性的评估指标上取得了最先进的性能。