In this paper we establish rigorous benchmarks for image classifier
robustness. Our first benchmark, ImageNet-C, standardizes and expands the
corruption robustness topic, while showing which classifiers are preferable in
safety-critical applications. Then we propose a new dataset called ImageNet-P
which enables researchers to benchmark a classifier's robustness to common
perturbations. Unlike recent robustness research, this benchmark evaluates
performance on common corruptions and perturbations not worst-case adversarial
perturbations. We find that there are negligible changes in relative corruption
robustness from AlexNet classifiers to ResNet classifiers. Afterward we
discover ways to enhance corruption and perturbation robustness. We even find
that a bypassed adversarial defense provides substantial common perturbation
robustness. Together our benchmarks may aid future work toward networks that
robustly generalize.

该文章为图像分类器的稳健性建立了严格的基准测试，并提出了两个基准测试 ImageNet-C 和 ImageNet-P，用于评估分类器对常见扰动和干扰的稳健性。研究发现，从 AlexNet 分类器到 ResNet 分类器，相对污染鲁棒性几乎没有变化，而绕过的对抗性防御提供了实质性的常见干扰强度。