There has been emerging interest in using transductive learning for
adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020;
Wang et al., ArXiv 2021). Compared to traditional defenses, these defense
mechanisms "dynamically learn" the model based on test-time input; and
theoretically, attacking these defenses reduces to solving a bilevel
optimization problem, which poses difficulty in crafting adaptive attacks. In
this paper, we examine these defense mechanisms from a principled threat
analysis perspective. We formulate and analyze threat models for
transductive-learning based defenses, and point out important subtleties. We
propose the principle of attacking model space for solving bilevel attack
objectives, and present Greedy Model Space Attack (GMSA), an attack framework
that can serve as a new baseline for evaluating transductive-learning based
defenses. Through systematic evaluation, we show that GMSA, even with weak
instantiations, can break previous transductive-learning based defenses, which
were resilient to previous attacks, such as AutoAttack. On the positive side,
we report a somewhat surprising empirical result of "transductive adversarial
training": Adversarially retraining the model using fresh randomness at the
test time gives a significant increase in robustness against attacks we
consider.

本文从威胁分析视角出发，对基于转导学习的防御机制进行了系统评估，并提出了攻击模型空间的原则，通过这一原则设计的攻击框架 Greedy Model Space Attack 可以作为评估转导学习防御机制的新基准。同时，本文发现在考虑的攻击方式下，重新训练模型可以显著提高对抗攻击的鲁棒性。