The presence of label noise often misleads the training of deep neural
networks. Departing from the recent literature which largely assumes the label
noise rate is only determined by the true label class, the errors in
human-annotated labels are more likely to be dependent on the difficulty levels
of tasks, resulting in settings with instance-dependent label noise. We first
provide evidences that the heterogeneous instance-dependent label noise is
effectively down-weighting the examples with higher noise rates in a
non-uniform way and thus causes imbalances, rendering the strategy of directly
applying methods for class-dependent label noise questionable. Built on a
recent work peer loss [24], we then propose and study the potentials of a
second-order approach that leverages the estimation of several covariance terms
defined between the instance-dependent noise rates and the Bayes optimal label.
We show that this set of second-order statistics successfully captures the
induced imbalances. We further proceed to show that with the help of the
estimated second-order statistics, we identify a new loss function whose
expected risk of a classifier under instance-dependent label noise is
equivalent to a new problem with only class-dependent label noise. This fact
allows us to apply existing solutions to handle this better-studied setting. We
provide an efficient procedure to estimate these second-order statistics
without accessing either ground truth labels or prior knowledge of the noise
rates. Experiments on CIFAR10 and CIFAR100 with synthetic instance-dependent
label noise and Clothing1M with real-world human label noise verify our
approach. Our implementation is available at this https URL

该研究提出一种二阶方法来处理深度神经网络训练中的标签噪音问题，使用此方法可以解决由任务难度引起的实例相关的噪音，提供一个新的损失函数来处理实例相关噪音，进而能够使用已有的类相关噪音的解决方案来处理该问题。