Training machine learning models to be robust against adversarial inputs poses seemingly insurmountable challenges. To better understand model robustness, we consider the underlying problem of learning robust representations. We develop a general definition of representation vulnerability that captures the maximum change of mutual information between the input and output distributions, under the worst-case input distribution perturbation. We prove a theorem that establishes a lower bound on the minimum adversarial risk that can be achieved for any downstream classifier based on this definition. We then propose an unsupervised learning method for obtaining intrinsically robust representations by maximizing the worst-case mutual information between input and output distributions. Experiments on downstream classification tasks and analyses of saliency maps support the robustness of the representations found using unsupervised learning with our training principle.

本文研究了在最坏情况下，表示脆弱性与最小敌对风险之间的下界关系，并提出了一种基于最坏情况下的相互信息最大化的无监督学习方法来获得内在具有鲁棒性的表示，并通过下游分类任务的实验证明了所得到的具有鲁棒性的表示。

通过最差情况的互信息最大化学习对抗鲁棒表现