Adversarial attacks pose significant threats to deploying state-of-the-art classifiers in safety-critical applications. Two classes of methods have emerged to address this issue: empirical defences and certified defences. Although certified defences come with robustness guarantees, empirical defences such as adversarial training enjoy much higher popularity among practitioners. In this paper, we systematically compare the standard and robust error of these two robust training paradigms across multiple computer vision tasks. We show that in most tasks and for both $\mathscr{l}_\infty$-ball and $\mathscr{l}_2$-ball threat models, certified training with convex relaxations suffers from worse standard and robust error than adversarial training. We further explore how the error gap between certified and adversarial training depends on the threat model and the data distribution. In particular, besides the perturbation budget, we identify as important factors the shape of the perturbation set and the implicit margin of the data distribution. We support our arguments with extensive ablations on both synthetic and image datasets.

本文介绍了在安全关键应用中，对抗攻击对部署最先进的分类器构成重大威胁；总体上，经验证的防御方法虽然具有鲁棒性保证，但是实践中的对抗训练比较受欢迎。我们系统性地比较了这两种鲁棒性训练方法在多个计算机视觉任务中的标准错误和鲁棒错误，结果表明，在大多数任务和威胁模型下，采用凸松弛的认证训练比采用对抗训练更容易带来标准错误和鲁棒错误。此外，我们还探讨了认证和对抗训练之间的错误差距如何依赖于威胁模型和数据分布，并且除了扰动预算外，我们还确定了扰动集的形状和数据分布的隐式边缘等重要因素。本文在合成和图像数据集上进行了大量消融实验，证明我们的观点。

使用凸松弛进行认证训练会影响鲁棒性能