Modern deep learning systems are data-hungry. Learning with web data is one
of the feasible solutions, but will introduce label noise inevitably, which can
hinder the performance of deep neural networks. Sample selection is an
effective way to deal with label noise. The key is to separate clean samples
based on some criterion. Previous methods pay more attention to the small loss
criterion where small-loss samples are regarded as clean ones. Nevertheless,
such a strategy relies on the learning dynamics of each data instance. Some
noisy samples are still memorized due to frequently occurring corrupted
learning patterns. To tackle this problem, a training-free surrogate model is
preferred, freeing from the effect of memorization. In this work, we propose to
leverage the vision-language surrogate model CLIP to filter noisy samples
automatically. CLIP brings external knowledge to facilitate the selection of
clean samples with its ability of text-image alignment. Furthermore, a margin
adaptive loss is designed to regularize the selection bias introduced by CLIP,
providing robustness to label noise. We validate the effectiveness of our
proposed method on both real-world and synthetic noisy datasets. Our method
achieves significant improvement without CLIP involved during the inference
stage.

利用视觉 - 语言替代模型 CLIP 自动过滤噪声样本，并设计了一种适应性边界损失来规范由 CLIP 引入的选择偏差，从而提供对标签噪声的鲁棒性。在真实和合成噪声数据集上验证了我们提出的方法的有效性。