Current studies on adversarial robustness mainly focus on aggregating local
robustness results from a set of data samples to evaluate and rank different
models. However, the local statistics may not well represent the true global
robustness of the underlying unknown data distribution. To address this
challenge, this paper makes the first attempt to present a new framework,
called GREAT Score , for global robustness evaluation of adversarial
perturbation using generative models. Formally, GREAT Score carries the
physical meaning of a global statistic capturing a mean certified attack-proof
perturbation level over all samples drawn from a generative model. For
finite-sample evaluation, we also derive a probabilistic guarantee on the
sample complexity and the difference between the sample mean and the true mean.
GREAT Score has several advantages: (1) Robustness evaluations using GREAT
Score are efficient and scalable to large models, by sparing the need of
running adversarial attacks. In particular, we show high correlation and
significantly reduced computation cost of GREAT Score when compared to the
attack-based model ranking on RobustBench (Croce,et. al. 2021). (2) The use of
generative models facilitates the approximation of the unknown data
distribution. In our ablation study with different generative adversarial
networks (GANs), we observe consistency between global robustness evaluation
and the quality of GANs. (3) GREAT Score can be used for remote auditing of
privacy-sensitive black-box models, as demonstrated by our robustness
evaluation on several online facial recognition services.

本文提出了一个称为 GREAT Score 的新框架，用于利用生成模型对对抗扰动进行全球鲁棒性评估。该方法具有高效性、可扩展性、适用于隐私敏感的黑盒模型等多个优点。