The memorization effect of deep neural network (DNN) plays a pivotal role in
many state-of-the-art label-noise learning methods. To exploit this property,
the early stopping trick, which stops the optimization at the early stage of
training, is usually adopted. Current methods generally decide the early
stopping point by considering a DNN as a whole. However, a DNN can be
considered as a composition of a series of layers, and we find that the latter
layers in a DNN are much more sensitive to label noise, while their former
counterparts are quite robust. Therefore, selecting a stopping point for the
whole network may make different DNN layers antagonistically affected each
other, thus degrading the final performance. In this paper, we propose to
separate a DNN into different parts and progressively train them to address
this problem. Instead of the early stopping, which trains a whole DNN all at
once, we initially train former DNN layers by optimizing the DNN with a
relatively large number of epochs. During training, we progressively train the
latter DNN layers by using a smaller number of epochs with the preceding layers
fixed to counteract the impact of noisy labels. We term the proposed method as
progressive early stopping (PES). Despite its simplicity, compared with the
early stopping, PES can help to obtain more promising and stable results.
Furthermore, by combining PES with existing approaches on noisy label training,
we achieve state-of-the-art performance on image classification benchmarks.

本论文提出一种分步训练深度神经网络的方法，称为渐进式提前停止（Progressive Early Stopping, PES），以对抗训练过程中的标签噪声。该方法可以显著提高图像分类质量，并结合现有标签噪声训练方法，达到了最新成果。