Data poisoning backdoor attacks can cause undesirable behaviors in large
language models (LLMs), and defending against them is of increasing importance.
Existing defense mechanisms often assume that only one type of trigger is
adopted by the attacker, while defending against multiple simultaneous and
independent trigger types necessitates general defense frameworks and is
relatively unexplored. In this paper, we propose Nested Product of
Experts(NPoE) defense framework, which involves a mixture of experts (MoE) as a
trigger-only ensemble within the PoE defense framework to simultaneously defend
against multiple trigger types. During NPoE training, the main model is trained
in an ensemble with a mixture of smaller expert models that learn the features
of backdoor triggers. At inference time, only the main model is used.
Experimental results on sentiment analysis, hate speech detection, and question
classification tasks demonstrate that NPoE effectively defends against a
variety of triggers both separately and in trigger mixtures. Due to the
versatility of the MoE structure in NPoE, this framework can be further
expanded to defend against other attack settings

数据污染后门攻击会对大型语言模型造成不良行为，我们提出了 Nested Product of Experts (NPoE) 防御框架，通过专业模型的混合实现对多个触发器类型的同时防御。实验结果表明，NPoE 能有效地防御各种类型的触发器。