As clean ImageNet accuracy nears its ceiling, the research community is increasingly more concerned about robust accuracy under distributional shifts. While a variety of methods have been proposed to robustify neural networks, these techniques often target models trained on ImageNet classification. At the same time, it is a common practice to use ImageNet pretrained backbones for downstream tasks such as object detection, semantic segmentation, and image classification from different domains. This raises a question: Can these robust image classifiers transfer robustness to downstream tasks? For object detection and semantic segmentation, we find that a vanilla Swin Transformer, a variant of Vision Transformer tailored for dense prediction tasks, transfers robustness better than Convolutional Neural Networks that are trained to be robust to the corrupted version of ImageNet. For CIFAR10 classification, we find that models that are robustified for ImageNet do not retain robustness when fully fine-tuned. These findings suggest that current robustification techniques tend to emphasize ImageNet evaluations. Moreover, network architecture is a strong source of robustness when we consider transfer learning.

当前 ImageNet 的准确率已接近其上限，研究界越来越关注分布偏移下的鲁棒准确率。本论文主要关注的是如何提高神经网络的鲁棒性，并考虑使用预训练的 ImageNet 模型在目标检测、语义分割、图像分类等不同领域的跨域应用。研究表明，对于目标检测和语义分割等任务，Vanilla Swin Transformer 的鲁棒性相比训练过 ImageNet 的卷积神经网络更强。但对于 CIFAR10分类任务，当前的鲁棒化技术存在局限性。因此，建立更强大的网络体系结构，加强模型在不同领域之间的迁移学习，是提高模型鲁棒性的重要方向。

ImageNet鲁棒性能是否能够传递至下游任务？