The landscape of available textual adversarial attacks keeps growing, posing
severe threats and raising concerns regarding the deep NLP system's integrity.
However, the crucial problem of defending against malicious attacks has only
drawn the attention of the NLP community. The latter is nonetheless
instrumental in developing robust and trustworthy systems. This paper makes two
important contributions in this line of search: (i) we introduce LAROUSSE, a
new framework to detect textual adversarial attacks and (ii) we introduce
STAKEOUT, a new benchmark composed of nine popular attack methods, three
datasets, and two pre-trained models. LAROUSSE is ready-to-use in production as
it is unsupervised, hyperparameter-free, and non-differentiable, protecting it
against gradient-based methods. Our new benchmark STAKEOUT allows for a robust
evaluation framework: we conduct extensive numerical experiments which
demonstrate that LAROUSSE outperforms previous methods, and which allows to
identify interesting factors of detection rate variations.

该研究提出了两个重要贡献：引入了 LAROUSSE 框架来检测文本对抗攻击，引入了 STAKEOUT 基准评估框架，其中包含了九种常用攻击方法、三个数据集和两个预训练模型。通过广泛的数值实验，我们证明 LAROUSSE 优于先前的方法，并且可以识别出有意义的检测率变化因素。