It is imperative to ensure the stability of every prediction made by a
language model; that is, a language's prediction should remain consistent
despite minor input variations, like word substitutions. In this paper, we
investigate the problem of certifying a language model's robustness against
Universal Text Perturbations (UTPs), which have been widely used in universal
adversarial attacks and backdoor attacks. Existing certified robustness based
on random smoothing has shown considerable promise in certifying the
input-specific text perturbations (ISTPs), operating under the assumption that
any random alteration of a sample's clean or adversarial words would negate the
impact of sample-wise perturbations. However, with UTPs, masking only the
adversarial words can eliminate the attack. A naive method is to simply
increase the masking ratio and the likelihood of masking attack tokens, but it
leads to a significant reduction in both certified accuracy and the certified
radius due to input corruption by extensive masking. To solve this challenge,
we introduce a novel approach, the superior prompt search method, designed to
identify a superior prompt that maintains higher certified accuracy under
extensive masking. Additionally, we theoretically motivate why ensembles are a
particularly suitable choice as base prompts for random smoothing. The method
is denoted by superior prompt ensembling technique. We also empirically confirm
this technique, obtaining state-of-the-art results in multiple settings. These
methodologies, for the first time, enable high certified accuracy against both
UTPs and ISTPs. The source code of CR-UTP is available at
this https URL

通过引入一种新的方法，即优越提示搜索方法，我们首次实现了针对 Universal Text Perturbations（UTPs）和 input-specific text perturbations (ISTPs) 的高可证明准确性。