Pre-trained contextualized language models (PrLMs) have led to strong
performance gains in downstream natural language understanding tasks. However,
PrLMs can still be easily fooled by adversarial word substitution, which is one
of the most challenging textual adversarial attack methods. Existing defence
approaches suffer from notable performance loss and complexities. Thus, this
paper presents a compact and performance-preserved framework, Anomaly Detection
with Frequency-Aware Randomization (ADFAR). In detail, we design an auxiliary
anomaly detection classifier and adopt a multi-task learning procedure, by
which PrLMs are able to distinguish adversarial input samples. Then, in order
to defend adversarial word substitution, a frequency-aware randomization
process is applied to those recognized adversarial input samples. Empirical
results show that ADFAR significantly outperforms those newly proposed defense
methods over various tasks with much higher inference speed. Remarkably, ADFAR
does not impair the overall performance of PrLMs. The code is available at
this https URL

本文提出了一种紧凑且性能不受影响的框架 ADFAR，采用辅助异常检测分类器和多任务学习过程较好地识别了对抗性输入的样本，并应用了频率感知的随机化过程防御对抗词置换攻击， 在各种任务上显着优于其他防御方法，且没有损害 PrLMs 的整体性能。