The presence of noisy labels in a training dataset can significantly impact the performance of machine learning models. To tackle this issue, researchers have explored methods for Learning with Noisy Labels to identify clean samples and reduce the influence of noisy labels. However, constraining the influence of a certain portion of the training dataset can result in a reduction in overall generalization performance. To alleviate this, recent studies have considered the careful utilization of noisy labels by leveraging huge computational resources. Therefore, the increasing training cost necessitates a reevaluation of efficiency. In other areas of research, there has been a focus on developing fine-tuning techniques for large pre-trained models that aim to achieve both high generalization performance and efficiency. However, these methods have mainly concentrated on clean datasets, and there has been limited exploration of the noisy label scenario. In this research, our aim is to find an appropriate way to fine-tune pre-trained models for noisy labeled datasets. To achieve this goal, we investigate the characteristics of pre-trained models when they encounter noisy datasets. Through empirical analysis, we introduce a novel algorithm called TURN, which robustly and efficiently transfers the prior knowledge of pre-trained models. The algorithm consists of two main steps: (1) independently tuning the linear classifier to protect the feature extractor from being distorted by noisy labels, and (2) reducing the noisy label ratio and fine-tuning the entire model based on the noise-reduced dataset to adapt it to the target dataset. The proposed algorithm has been extensively tested and demonstrates efficient yet improved denoising performance on various benchmarks compared to previous methods.

在这项研究中，我们的目标是找到一种适用于带有噪声标签数据集的预训练模型微调的合适方法。通过经验分析，我们引入了一种名为TURN的新算法，其能够稳健且高效地传递预训练模型的先验知识。该算法包括两个主要步骤：（1）独立调整线性分类器来保护特征提取器免受噪声标签的扭曲影响，以及（2）减少噪声标签比例并基于降噪后的数据集对整个模型进行微调以适应目标数据集。与先前方法相比，该算法在各种基准测试中广泛测试，表现出高效且改进的降噪性能。

对预训练模型进行微调以提高在有噪标签下的稳健性