Although large language models (LLMs) have achieved great success in vast
real-world applications, their vulnerabilities towards noisy inputs have
significantly limited their uses, especially in high-stake environments. In
these contexts, it is crucial to ensure that every prediction made by large
language models is stable, i.e., LLM predictions should be consistent given
minor differences in the input. This largely falls into the study of certified
robust LLMs, i.e., all predictions of LLM are certified to be correct in a
local region around the input. Randomized smoothing has demonstrated great
potential in certifying the robustness and prediction stability of LLMs.
However, randomized smoothing requires adding noise to the input before model
prediction, and its certification performance depends largely on the model's
performance on corrupted data. As a result, its direct application to LLMs
remains challenging and often results in a small certification radius. To
address this issue, we take advantage of the multitasking nature of LLMs and
propose to denoise the corrupted inputs with LLMs in a self-denoising manner.
Different from previous works like denoised smoothing, which requires training
a separate model to robustify LLM, our method enjoys far better efficiency and
flexibility. Our experiment results show that our method outperforms the
existing certification methods under both certified robustness and empirical
robustness. The codes are available at
this https URL

本文提出了一种新的针对大型语言模型的认证健壮方法，提高了模型的认证健壮性和预测稳定性，比现有认证方法表现更优秀，实验结果表明该方法在经验证健壮性和实证健壮性方面均优于现有认证方法。