Deep neural networks have incredible capacity and expressibility, and can
seemingly memorize any training set. This introduces a problem when training in
the presence of noisy labels, as the noisy examples cannot be distinguished
from clean examples by the end of training. Recent research has dealt with this
challenge by utilizing the fact that deep networks seem to memorize clean
examples much earlier than noisy examples. Here we report a new empirical
result: for each example, when looking at the time it has been memorized by
each model in an ensemble of networks, the diversity seen in noisy examples is
much larger than the clean examples. We use this observation to develop a new
method for noisy labels filtration. The method is based on a statistics of the
data, which captures the differences in ensemble learning dynamics between
clean and noisy data. We test our method on three tasks: (i) noise amount
estimation; (ii) noise filtration; (iii) supervised classification. We show
that our method improves over existing baselines in all three tasks using a
variety of datasets, noise models, and noise levels. Aside from its improved
performance, our method has two other advantages. (i) Simplicity, which implies
that no additional hyperparameters are introduced. (ii) Our method is modular:
it does not work in an end-to-end fashion, and can therefore be used to clean a
dataset for any other future usage.

利用深度神经网络记忆清晰样本更早的现象，发展一种统计学方法，根据集成学习动态数据中干净和嘈杂数据的差异，改善了噪声标记估计，噪声过滤和指导分类的方法。