Deep neural network-based classifiers trained with the categorical
cross-entropy (CCE) loss are sensitive to label noise in the training data. One
common type of method that can mitigate the impact of label noise can be viewed
as supervised robust methods; one can simply replace the CCE loss with a loss
that is robust to label noise, or re-weight training samples and down-weight
those with higher loss values. Recently, another type of method using
semi-supervised learning (SSL) has been proposed, which augments these
supervised robust methods to exploit (possibly) noisy samples more effectively.
Although supervised robust methods perform well across different data types,
they have been shown to be inferior to the SSL methods on image classification
tasks under label noise. Therefore, it remains to be seen that whether these
supervised robust methods can also perform well if they can utilize the
unlabeled samples more effectively. In this paper, we show that by initializing
supervised robust methods using representations learned through contrastive
learning leads to significantly improved performance under label noise.
Surprisingly, even the simplest method (training a classifier with the CCE
loss) can outperform the state-of-the-art SSL method by more than 50\% under
high label noise when initialized with contrastive learning. Our implementation
will be publicly available at
{https://github.com/arghosh/noisy_label_pretrain}.

本论文讨论如何提高基于深度神经网络分类器在训练数据中鲁棒性以解决标签噪声问题，通过代替分类交叉熵损失为适应标签噪声的损失函数，或者改变训练样本的权重来缓解标签噪声的影响，并介绍一种基于对比学习技术来初始化有监督鲁棒方法以解决高标签噪声下的图像分类问题的方法。

对比学习提高模型在标签噪声下的鲁棒性

Contrastive Learning Improves Model Robustness Under Label Noise

Robust speaker recognition, including in the presence of malicious attacks,
is becoming increasingly important and essential, especially due to the
proliferation of several smart speakers and personal agents that interact with
an individual's voice commands to perform diverse, and even sensitive tasks.
Adversarial attack is a recently revived domain which is shown to be effective
in breaking deep neural network-based classifiers, specifically, by forcing
them to change their posterior distribution by only perturbing the input
samples by a very small amount. Although, significant progress in this realm
has been made in the computer vision domain, advances within speaker
recognition is still limited. The present expository paper considers several
state-of-the-art adversarial attacks to a deep speaker recognition system,
employing strong defense methods as countermeasures, and reporting on several
ablation studies to obtain a comprehensive understanding of the problem. The
experiments show that the speaker recognition systems are vulnerable to
adversarial attacks, and the strongest attacks can reduce the accuracy of the
system from 94% to even 0%. The study also compares the performances of the
employed defense methods in detail, and finds adversarial training based on
Projected Gradient Descent (PGD) to be the best defense method in our setting.
We hope that the experiments presented in this paper provide baselines that can
be useful for the research community interested in further studying adversarial
robustness of speaker recognition systems.

本文研究了深度说话人识别系统所面临的对抗攻击问题，尝试了多种防御方法，并且实验证明了对抗攻击可能会导致准确度降至 0%，并且发现了以投影梯度下降为基础的对抗训练方法是最好的防御手段。