State-of-the-art few-shot learning (FSL) methods leverage prompt-based fine-tuning to obtain remarkable results for natural language understanding (NLU) tasks. While much of the prior FSL methods focus on improving downstream task performance, there is a limited understanding of the adversarial robustness of such methods. In this work, we conduct an extensive study of several state-of-the-art FSL methods to assess their robustness to adversarial perturbations. To better understand the impact of various factors towards robustness (or the lack of it), we evaluate prompt-based FSL methods against fully fine-tuned models for aspects such as the use of unlabeled data, multiple prompts, number of few-shot examples, model size and type. Our results on six GLUE tasks indicate that compared to fully fine-tuned models, vanilla FSL methods lead to a notable relative drop in task performance (i.e., are less robust) in the face of adversarial perturbations. However, using (i) unlabeled data for prompt-based FSL and (ii) multiple prompts flip the trend. We further demonstrate that increasing the number of few-shot examples and model size lead to increased adversarial robustness of vanilla FSL methods. Broadly, our work sheds light on the adversarial robustness evaluation of prompt-based FSL methods for NLU tasks.

本篇研究对几种最先进的小样本学习方法进行了广泛的研究，以评估它们对敌对扰动的鲁棒性，结果表明相对于完全微调模型，普通的Few-shot学习方法在面对敌对扰动时表现出显著的下降，但使用（i）用于提示的未标记数据和（ii）多个提示扭转了这种趋势，并且增加少量样本的例数和模型大小可以增加其鲁棒性。

基于Prompt的少样本学习在自然语言理解中对抗鲁棒性研究