The language models, especially the basic text classification models, have
been shown to be susceptible to textual adversarial attacks such as synonym
substitution and word insertion attacks. To defend against such attacks, a
growing body of research has been devoted to improving the model robustness.
However, providing provable robustness guarantees instead of empirical
robustness is still widely unexplored. In this paper, we propose Text-CRS, a
generalized certified robustness framework for natural language processing
(NLP) based on randomized smoothing. To our best knowledge, existing certified
schemes for NLP can only certify the robustness against $\ell_0$ perturbations
in synonym substitution attacks. Representing each word-level adversarial
operation (i.e., synonym substitution, word reordering, insertion, and
deletion) as a combination of permutation and embedding transformation, we
propose novel smoothing theorems to derive robustness bounds in both
permutation and embedding space against such adversarial operations. To further
improve certified accuracy and radius, we consider the numerical relationships
between discrete words and select proper noise distributions for the randomized
smoothing. Finally, we conduct substantial experiments on multiple language
models and datasets. Text-CRS can address all four different word-level
adversarial operations and achieve a significant accuracy improvement. We also
provide the first benchmark on certified accuracy and radius of four word-level
operations, besides outperforming the state-of-the-art certification against
synonym substitution attacks.

本文提出了一种基于随机平滑的泛化认证鲁棒性框架 Text-CRS，用于自然语言处理中的文本，并对单词级对抗操作进行了证明，实现了显著的准确性提升。