In this appraisal paper, we evaluate the efficacy of SHIELD, a
compression-based defense framework for countering adversarial attacks on image
classification models, which was published at KDD 2018. Here, we consider
alternative threat models not studied in the original work, where we assume
that an adaptive adversary is aware of the ensemble defense approach, the
defensive pre-processing, and the architecture and weights of the models used
in the ensemble. We define scenarios with varying levels of threat and
empirically analyze the proposed defense by varying the degree of information
available to the attacker, spanning from a full white-box attack to the
gray-box threat model described in the original work. To evaluate the
robustness of the defense against an adaptive attacker, we consider the
targeted-attack success rate of the Projected Gradient Descent (PGD) attack,
which is a strong gradient-based adversarial attack proposed in adversarial
machine learning research. We also experiment with training the SHIELD ensemble
from scratch, which is different from re-training using a pre-trained model as
done in the original work. We find that the targeted PGD attack has a success
rate of 64.3% against the original SHIELD ensemble in the full white box
scenario, but this drops to 48.9% if the models used in the ensemble are
trained from scratch instead of being retrained. Our experiments further reveal
that an ensemble whose models are re-trained indeed have higher correlation in
the cosine similarity space, and models that are trained from scratch are less
vulnerable to targeted attacks in the white-box and gray-box scenarios.

本文评估用于对抗敌对攻击的压缩式防御框架 SHIELD 的效力，并在原有工作的基础上考虑了替代威胁模型，提出了具有不同危险程度的情况，并通过实验结果得出了在白盒和灰盒情景下训练模型的相关性与承受目标攻击成功率的相关性，证明了从零开始训练模型的更强鲁棒性。