Deep neural networks have been shown to be very powerful modeling tools for
many supervised learning tasks involving complex input patterns. However, they
can also easily overfit to training set biases and label noises. In addition to
various regularizers, example reweighting algorithms are popular solutions to
these problems, but they require careful tuning of additional hyperparameters,
such as example mining schedules and regularization hyperparameters. In
contrast to past reweighting methods, which typically consist of functions of
the cost value of each example, in this work we propose a novel meta-learning
algorithm that learns to assign weights to training examples based on their
gradient directions. To determine the example weights, our method performs a
meta gradient descent step on the current mini-batch example weights (which are
initialized from zero) to minimize the loss on a clean unbiased validation set.
Our proposed method can be easily implemented on any type of deep network, does
not require any additional hyperparameter tuning, and achieves impressive
performance on class imbalance and corrupted label problems where only a small
amount of clean validation data is available.

该研究提出了一种基于元学习和梯度下降的新算法，来对训练样本进行权重分配，从而达到解决深度神经网络在训练集偏差和标签噪声方面的过度拟合问题，而不需要额外的超参数调整，并在少量有干净核准数据的情况下，实现了在类别不平衡和标签污染问题上的出色性能。