We leverage diffusion models to study the robustness-performance tradeoff of
robust classifiers. Our approach introduces a simple, pretrained diffusion
method to generate low-norm counterfactual examples (CEs): semantically altered
data which results in different true class membership. We report that the
confidence and accuracy of robust models on their clean training data are
associated with the proximity of the data to their CEs. Moreover, robust models
perform very poorly when evaluated on the CEs directly, as they become
increasingly invariant to the low-norm, semantic changes brought by CEs. The
results indicate a significant overlap between non-robust and semantic
features, countering the common assumption that non-robust features are not
interpretable.

我们利用扩散模型研究了鲁棒分类器的鲁棒性和性能之间的权衡。我们的方法引入了一个简单的预训练扩散方法来生成低范数的反事实例：语义上改变的数据导致不同的真实类别成员身份。我们报告在干净的训练数据上，鲁棒模型的置信度和准确性与数据与反事实例的接近程度相关。此外，当直接在反事实例上进行评估时，鲁棒模型的性能非常差，因为它们对反事实例通过低范数和语义变化而变得越来越不变。结果表明，非鲁棒和语义特征之间存在显著的重叠，与普遍的非鲁棒特征不可解释的假设相反。