Adversarial training is a popular defense strategy against attack threat
models with bounded Lp norms. However, it often degrades the model performance
on normal images and the defense does not generalize well to novel attacks.
Given the success of deep generative models such as GANs and VAEs in
characterizing the underlying manifold of images, we investigate whether or not
the aforementioned problems can be remedied by exploiting the underlying
manifold information. To this end, we construct an "On-Manifold ImageNet"
(OM-ImageNet) dataset by projecting the ImageNet samples onto the manifold
learned by StyleGSN. For this dataset, the underlying manifold information is
exact. Using OM-ImageNet, we first show that adversarial training in the latent
space of images improves both standard accuracy and robustness to on-manifold
attacks. However, since no out-of-manifold perturbations are realized, the
defense can be broken by Lp adversarial attacks. We further propose Dual
Manifold Adversarial Training (DMAT) where adversarial perturbations in both
latent and image spaces are used in robustifying the model. Our DMAT improves
performance on normal images, and achieves comparable robustness to the
standard adversarial training against Lp attacks. In addition, we observe that
models defended by DMAT achieve improved robustness against novel attacks which
manipulate images by global color shifts or various types of image filtering.
Interestingly, similar improvements are also achieved when the defended models
are tested on out-of-manifold natural images. These results demonstrate the
potential benefits of using manifold information in enhancing robustness of
deep learning models against various types of novel adversarial attacks.

通过在图像的潜在空间中对对抗样本进行对抗训练以及利用生成模型中学习到的流形信息进行双流形对抗训练，可以大大提高深度学习模型的鲁棒性，从而有效地应对多种新颖的对抗攻击。

双流形对抗性鲁棒性：抵御 Lp 和非 Lp 对抗攻击

Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp  Adversarial Attacks

Most existing adversarial defenses only measure robustness to L_p adversarial
attacks. Not only are adversaries unlikely to exclusively create small L_p
perturbations, adversaries are unlikely to remain fixed. Adversaries adapt and
evolve their attacks; hence adversarial defenses must be robust to a broad
range of unforeseen attacks. We address this discrepancy between research and
reality by proposing a new evaluation framework called ImageNet-UA. Our
framework enables the research community to test ImageNet model robustness
against attacks not encountered during training. To create ImageNet-UA's
diverse attack suite, we introduce a total of four novel adversarial attacks.
We also demonstrate that, in comparison to ImageNet-UA, prevailing L_inf
robustness assessments give a narrow account of model robustness. By evaluating
current defenses with ImageNet-UA, we find they provide little robustness to
unforeseen attacks. We hope the greater variety and realism of ImageNet-UA
enables development of more robust defenses which can generalize beyond attacks
seen during training.

提出了一种名为 ImageNet-UA 的模型鲁棒性测试框架，旨在测试模型对未遭遇过的对抗攻击的鲁棒性。该框架引入了四种新的对抗攻击，与 L_inf 等现有鲁棒性评估方法相比，ImageNet-UA 能更好地展现模型的鲁棒性。目前的防御措施在面对未预料的攻击时提供很少的保障，期望使用更加多样化和现实的 ImageNet-UA 能有助于开发泛化性更好的防御措施。