Learning with label dependent label noise has been extensively explored in
both theory and practice; however, dealing with instance (i.e., feature) and
label dependent label noise continues to be a challenging task. The difficulty
arises from the fact that the noise rate varies for each instance, making it
challenging to estimate accurately. The question of whether it is possible to
learn a reliable model using only noisy samples remains unresolved. We answer
this question with a theoretical analysis that provides matching upper and
lower bounds. Surprisingly, our results show that, without any additional
assumptions, empirical risk minimization achieves the optimal excess risk
bound. Specifically, we derive a novel excess risk bound proportional to the
noise level, which holds in very general settings, by comparing the empirical
risk minimizers obtained from clean samples and noisy samples. Second, we show
that the minimax lower bound for the 0-1 loss is a constant proportional to the
average noise rate. Our findings suggest that learning solely with noisy
samples is impossible without access to clean samples or strong assumptions on
the distribution of the data.

本文通过理论分析探讨了在样本噪声存在的情况下，仅使用噪声样本能否学习到可靠模型的问题。作者认为，没有额外假设条件的情况下，经验风险最小化可以达到最优风险上限。此外，文章还讨论了 0-1 损失的极小极大下限问题，认为纯使用噪声样本无法学习。

基于实例和标签相关的标签噪声的二分类

Binary Classification with Instance and Label Dependent Label Noise

Suppose we have a sample of instances paired with binary labels corrupted by
arbitrary instance- and label-dependent noise. With sufficiently many such
samples, can we optimally classify and rank instances with respect to the
noise-free distribution? We provide a theoretical analysis of this question,
with three main contributions. First, we prove that for instance-dependent
noise, any algorithm that is consistent for classification on the noisy
distribution is also consistent on the clean distribution. Second, we prove
that for a broad class of instance- and label-dependent noise, a similar
consistency result holds for the area under the ROC curve. Third, for the
latter noise model, when the noise-free class-probability function belongs to
the generalised linear model family, we show that the Isotron can efficiently
and provably learn from the corrupted sample.

对于数据集存在的任意的基于实例和标签的噪声，本文提出算法来使得能够对数据集进行最优化的分类和排名，同时给出三个关键结果，即对于基于实例的噪声和广泛的基于实例和标签的噪声类，可以在噪声分布上保持一致性，而 Isotron 可以有效地从污染的样本中学习。