Adversarial examples are malicious inputs to machine learning models that
trigger a misclassification. This type of attack has been studied for close to
a decade, and we find that there is a lack of study and formalization of
adversary knowledge when mounting attacks. This has yielded a complex space of
attack research with hard-to-compare threat models and attacks. We focus on the
image classification domain and provide a theoretical framework to study
adversary knowledge inspired by work in order theory. We present an adversarial
example game, inspired by cryptographic games, to standardize attacks. We
survey recent attacks in the image classification domain and classify their
adversary's knowledge in our framework. From this systematization, we compile
results that both confirm existing beliefs about adversary knowledge, such as
the potency of information about the attacked model as well as allow us to
derive new conclusions on the difficulty associated with the white-box and
transferable threat models, for example, that transferable attacks might not be
as difficult as previously thought.

对于机器学习模型而言，对抗性样本是指触发误分类的恶意输入。我们发现当进行攻击时，对抗者的知识缺乏研究和形式化，导致攻击研究领域在威胁模型和攻击方法上非常复杂且难以比较。本研究在图像分类领域提供了一个受序理论启发的理论框架来研究对抗者的知识，同时提出了一个受密码学游戏启发的对抗性样本游戏来标准化攻击。通过对最近的图像分类领域攻击进行整理和分类，我们从中总结出攻击者的知识，既验证了关于对抗者知识的现有观点，如受攻击模型信息的重要性，也得出了新的结论，例如白盒和可传递威胁模型的困难程度可能没有之前想象的那么高。