In the field of natural language processing, the prevalent approach involves fine-tuning pretrained language models (PLMs) using local samples. Recent research has exposed the susceptibility of PLMs to backdoor attacks, wherein the adversaries can embed malicious prediction behaviors by manipulating a few training samples. In this study, our objective is to develop a backdoor-resistant tuning procedure that yields a backdoor-free model, no matter whether the fine-tuning dataset contains poisoned samples. To this end, we propose and integrate a honeypot module into the original PLM, specifically designed to absorb backdoor information exclusively. Our design is motivated by the observation that lower-layer representations in PLMs carry sufficient backdoor features while carrying minimal information about the original tasks. Consequently, we can impose penalties on the information acquired by the honeypot module to inhibit backdoor creation during the fine-tuning process of the stem network. Comprehensive experiments conducted on benchmark datasets substantiate the effectiveness and robustness of our defensive strategy. Notably, these results indicate a substantial reduction in the attack success rate ranging from 10\% to 40\% when compared to prior state-of-the-art methods.

本研究旨在设计一种抵御后门攻击的微调过程，使用预训练语言模型并引入蜜罐模块，以吸收后门信息，并通过对中间层表示的约束，在微调过程中防止后门创建。在基准数据集上进行的综合实验证明了我们的防御策略的有效性和鲁棒性，与先前的最新方法相比，攻击成功率大幅降低了10％至40％。

陷阱设置：通过诱饵捕获和击败预训练语言模型中的后门