The improvement of language model robustness, including successful defense
against adversarial attacks, remains an open problem. In computer vision
settings, the stochastic noising and de-noising process provided by diffusion
models has proven useful for purifying input images, thus improving model
robustness against adversarial attacks. Similarly, some initial work has
explored the use of random noising and de-noising to mitigate adversarial
attacks in an NLP setting, but improving the quality and efficiency of these
methods is necessary for them to remain competitive. We extend upon methods of
input text purification that are inspired by diffusion processes, which
randomly mask and refill portions of the input text before classification. Our
novel method, MaskPure, exceeds or matches robustness compared to other
contemporary defenses, while also requiring no adversarial classifier training
and without assuming knowledge of the attack type. In addition, we show that
MaskPure is provably certifiably robust. To our knowledge, MaskPure is the
first stochastic-purification method with demonstrated success against both
character-level and word-level attacks, indicating the generalizable and
promising nature of stochastic denoising defenses. In summary: the MaskPure
algorithm bridges literature on the current strongest certifiable and empirical
adversarial defense methods, showing that both theoretical and practical
robustness can be obtained together. Code is available on GitHub at
this https URL

通过扩展受扩散过程启发的输入文本纯化方法，我们提出了一种名为 MaskPure 的新方法，无需对抗分类器进行训练且不需要攻击类型知识，它在同类防御方法中显示出卓越的稳健性和可靠性，同时证明了它的可证实稳健性。