Adversarial robustness is a research area that has recently received a lot of
attention in the quest for trustworthy artificial intelligence. However, recent
works on adversarial robustness have focused on supervised learning where it is
assumed that labeled data is plentiful. In this paper, we investigate
semi-supervised adversarial training where labeled data is scarce. We derive
two upper bounds for the robust risk and propose a regularization term for
unlabeled data motivated by these two upper bounds. Then, we develop a
semi-supervised adversarial training algorithm that combines the proposed
regularization term with knowledge distillation using a semi-supervised teacher
(i.e., a teacher model trained using a semi-supervised learning algorithm). Our
experiments show that our proposed algorithm achieves state-of-the-art
performance with significant margins compared to existing algorithms. In
particular, compared to supervised learning algorithms, performance of our
proposed algorithm is not much worse even when the amount of labeled data is
very small. For example, our algorithm with only 8\% labeled data is comparable
to supervised adversarial training algorithms that use all labeled data, both
in terms of standard and robust accuracies on CIFAR-10.

关键词：对抗鲁棒性、半监督对抗训练、有标签数据稀缺、正则化项、知识蒸馏。摘要：本文研究了对抗鲁棒性的半监督对抗训练，探究了稀缺有标签数据情况下的两个鲁棒风险上界，并提出了正则化项以应用于无标签数据，最后通过半监督教师模型的知识蒸馏结合提出的正则化项进行半监督对抗训练算法。实验证明，相较于现有算法，我们提出的算法在性能方面取得了显著的提升，并且即使有限的有标签数据量，我们的算法仍能与使用所有标签数据的监督对抗训练算法相媲美，无论是在标准准确率还是在鲁棒准确率上，比如我们的算法只使用 8％的有标签数据时，在 CIFAR-10 数据集上与监督对抗训练算法相比表现接近。

通过自适应加权正则化和知识蒸馏增强低标签模式下的对抗鲁棒性

Enhancing Adversarial Robustness in Low-Label Regime via Adaptively  Weighted Regularization and Knowledge Distillation

In cross-lingual text classification, one seeks to exploit labeled data from
one language to train a text classification model that can then be applied to a
completely different language. Recent multilingual representation models have
made it much easier to achieve this. Still, there may still be subtle
differences between languages that are neglected when doing so. To address
this, we present a semi-supervised adversarial training process that minimizes
the maximal loss for label-preserving input perturbations. The resulting model
then serves as a teacher to induce labels for unlabeled target language samples
that can be used during further adversarial training, allowing us to gradually
adapt our model to the target language. Compared with a number of strong
baselines, we observe significant gains in effectiveness on document and intent
classification for a diverse set of languages.

本研究提出了一种半监督敌对训练过程，通过最小化标签保留输入扰动的最大损失来改进跨语言文本分类，进而为目标语言样本诱导标签以进一步适应模型。与多个强基线比较，我们观察到了在各种语言的文档和意图分类上的显著增益。