Supervised deep learning depends on massive accurately annotated examples,
which is usually impractical in many real-world scenarios. A typical
alternative is learning from multiple noisy annotators. Numerous earlier works
assume that all labels are noisy, while it is usually the case that a few
trusted samples with clean labels are available. This raises the following
important question: how can we effectively use a small amount of trusted data
to facilitate robust classifier learning from multiple annotators? This paper
proposes a data-efficient approach, called \emph{Trustable Co-label Learning}
(TCL), to learn deep classifiers from multiple noisy annotators when a small
set of trusted data is available. This approach follows the coupled-view
learning manner, which jointly learns the data classifier and the label
aggregator. It effectively uses trusted data as a guide to generate trustable
soft labels (termed co-labels). A co-label learning can then be performed by
alternately reannotating the pseudo labels and refining the classifiers. In
addition, we further improve TCL for a special complete data case, where each
instance is labeled by all annotators and the label aggregator is represented
by multilayer neural networks to enhance model capacity. Extensive experiments
on synthetic and real datasets clearly demonstrate the effectiveness and
robustness of the proposed approach. Source code is available at
this https URL

该研究提出了一种数据高效的方法，称为 “可信协同学习”（TCL），在多个嘈杂标注器的情况下从多个受信任的数据中学习深度分类器。这种方法通过联合学习数据分类器和标签聚合器来有效地使用受信任的数据生成可信软标签（称为共标签）。

基于多个有噪音注释者的可信合作标记学习

Trustable Co-label Learning from Multiple Noisy Annotators

The growing importance of massive datasets used for deep learning makes
robustness to label noise a critical property for classifiers to have. Sources
of label noise include automatic labeling, non-expert labeling, and label
corruption by data poisoning adversaries. Numerous previous works assume that
no source of labels can be trusted. We relax this assumption and assume that a
small subset of the training data is trusted. This enables substantial label
corruption robustness performance gains. In addition, particularly severe label
noise can be combated by using a set of trusted data with clean labels. We
utilize trusted data by proposing a loss correction technique that utilizes
trusted examples in a data-efficient manner to mitigate the effects of label
noise on deep neural network classifiers. Across vision and natural language
processing tasks, we experiment with various label noises at several strengths,
and show that our method significantly outperforms existing methods.

本文提出了一种在深度神经网络分类器中使用有信任子集数据以及基于损失修正技术的方法，大大提高了分类器对标签噪声的鲁棒性。实验结果表明，该方法在视觉和自然语言处理任务中均取得了较好的性能表现。