ImageNet pre-training has enabled state-of-the-art results on many tasks. In
spite of its recognized contribution to generalization, we observed in this
study that ImageNet pre-training also transfers adversarial non-robustness from
pre-trained model into fine-tuned model in the downstream classification tasks.
We first conducted experiments on various datasets and network backbones to
uncover the adversarial non-robustness in fine-tuned model. Further analysis
was conducted on examining the learned knowledge of fine-tuned model and
standard model, and revealed that the reason leading to the non-robustness is
the non-robust features transferred from ImageNet pre-trained model. Finally,
we analyzed the preference for feature learning of the pre-trained model,
explored the factors influencing robustness, and introduced a simple robust
ImageNet pre-training solution. Our code is available at
https://github.com/jiamingzhang94/ImageNet-Pretraining-transfers-non-robustness.

本文研究表明，ImageNet 预训练模型虽然在泛化方面作出了贡献，但同时也将非健壮性特性从预训练模型转移到了下游分类任务中的微调模型，本文介绍了其原因并提出了一种简单的健壮性 ImageNet 预训练解决方案。