Learning with noisy labels has become an effective strategy for enhancing the
robustness of models, which enables models to better tolerate inaccurate data.
Existing methods either focus on optimizing the loss function to mitigate the
interference from noise, or design procedures to detect potential noise and
correct errors. However, their effectiveness is often compromised in
representation learning due to the dilemma where models overfit to noisy
labels. To address this issue, this paper proposes a relation modeling and
distillation framework that models inter-sample relationships via
self-supervised learning and employs knowledge distillation to enhance
understanding of latent associations, which mitigate the impact of noisy
labels. Specifically, the proposed method, termed RMDNet, includes two main
modules, where the relation modeling (RM) module implements the contrastive
learning technique to learn representations of all data, an unsupervised
approach that effectively eliminates the interference of noisy tags on feature
extraction. The relation-guided representation learning (RGRL) module utilizes
inter-sample relation learned from the RM module to calibrate the
representation distribution for noisy samples, which is capable of improving
the generalization of the model in the inference phase. Notably, the proposed
RMDNet is a plug-and-play framework that can integrate multiple methods to its
advantage. Extensive experiments were conducted on two datasets, including
performance comparison, ablation study, in-depth analysis and case study. The
results show that RMDNet can learn discriminative representations for noisy
data, which results in superior performance than the existing methods.

通过自我监督学习建模样本之间的关系，并采用知识蒸馏的方法增强对潜在关联的理解，从而缓解模糊标签的影响，该研究提出了一个处理噪声标签的关系建模和蒸馏框架，该框架能够学习具有噪声的数据的判别性表示，并取得比现有方法更优越的性能。