Machine-learning models are known to be vulnerable to evasion attacks that
perturb model inputs to induce misclassifications. In this work, we identify
real-world scenarios where the true threat cannot be assessed accurately by
existing attacks. Specifically, we find that conventional metrics measuring
targeted and untargeted robustness do not appropriately reflect a model's
ability to withstand attacks from one set of source classes to another set of
target classes. To address the shortcomings of existing methods, we formally
define a new metric, termed group-based robustness, that complements existing
metrics and is better-suited for evaluating model performance in certain attack
scenarios. We show empirically that group-based robustness allows us to
distinguish between models' vulnerability against specific threat models in
situations where traditional robustness metrics do not apply. Moreover, to
measure group-based robustness efficiently and accurately, we 1) propose two
loss functions and 2) identify three new attack strategies. We show empirically
that with comparable success rates, finding evasive samples using our new loss
functions saves computation by a factor as large as the number of targeted
classes, and finding evasive samples using our new attack strategies saves time
by up to 99\% compared to brute-force search methods. Finally, we propose a
defense method that increases group-based robustness by up to 3.52$\times$.

本文旨在通过定义一种补充现有度量方式的量化指标 —— 群体韧性，更好地评估模型在特定攻击情境下的表现并提出了两种损失函数、三种新的攻击策略，证明了该指标对于区分对特定威胁模型的脆弱性有更好的效果，并提出了一种防御方法，可以将群体韧性提高 3.52 倍。