Learning with noisy labels (LNL) has been extensively studied, with existing
approaches typically following a framework that alternates between clean sample
selection and semi-supervised learning (SSL). However, this approach has a
limitation: the clean set selected by the Deep Neural Network (DNN) classifier,
trained through self-training, inevitably contains noisy samples. This mixture
of clean and noisy samples leads to misguidance in DNN training during SSL,
resulting in impaired generalization performance due to confirmation bias
caused by error accumulation in sample selection. To address this issue, we
propose a method called Collaborative Sample Selection (CSS), which leverages
the large-scale pre-trained model CLIP. CSS aims to remove the mixed noisy
samples from the identified clean set. We achieve this by training a
2-Dimensional Gaussian Mixture Model (2D-GMM) that combines the probabilities
from CLIP with the predictions from the DNN classifier. To further enhance the
adaptation of CLIP to LNL, we introduce a co-training mechanism with a
contrastive loss in semi-supervised learning. This allows us to jointly train
the prompt of CLIP and the DNN classifier, resulting in improved feature
representation, boosted classification performance of DNNs, and reciprocal
benefits to our Collaborative Sample Selection. By incorporating auxiliary
information from CLIP and utilizing prompt fine-tuning, we effectively
eliminate noisy samples from the clean set and mitigate confirmation bias
during training. Experimental results on multiple benchmark datasets
demonstrate the effectiveness of our proposed method in comparison with the
state-of-the-art approaches.

利用 CLIP 模型的协作样本选择及预训练，并通过对 prompt 的微调以及协同训练 DNN 分类器，解决在学习有噪声标签的过程中由于样本选择错误累积导致的 DNN 训练偏见和泛化性能问题。

使用协作样本选择和对比半监督学习来学习带噪标签的内容

Learning with Noisy Labels Using Collaborative Sample Selection and  Contrastive Semi-Supervised Learning

A noisy training set usually leads to the degradation of the generalization
and robustness of neural networks. In this paper, we propose a novel
theoretically guaranteed clean sample selection framework for learning with
noisy labels. Specifically, we first present a Scalable Penalized Regression
(SPR) method, to model the linear relation between network features and one-hot
labels. In SPR, the clean data are identified by the zero mean-shift parameters
solved in the regression model. We theoretically show that SPR can recover
clean data under some conditions. Under general scenarios, the conditions may
be no longer satisfied; and some noisy data are falsely selected as clean data.
To solve this problem, we propose a data-adaptive method for Scalable Penalized
Regression with Knockoff filters (Knockoffs-SPR), which is provable to control
the False-Selection-Rate (FSR) in the selected clean data. To improve the
efficiency, we further present a split algorithm that divides the whole
training set into small pieces that can be solved in parallel to make the
framework scalable to large datasets. While Knockoffs-SPR can be regarded as a
sample selection module for a standard supervised training pipeline, we further
combine it with a semi-supervised algorithm to exploit the support of noisy
data as unlabeled data. Experimental results on several benchmark datasets and
real-world noisy datasets show the effectiveness of our framework and validate
the theoretical results of Knockoffs-SPR. Our code and pre-trained models will
be released.

本文提出了一种用于学习带噪标签的理论上保证的干净样本选择框架，通过 Scalable Penalized Regression（SPR）方法建模网络特征与 one-hot 标签之间的线性关系来识别干净数据，并通过数据自适应方法的 Knockoffs-SPR 控制假阳率，结果表明该方法有效。