Backdoors implanted in pre-trained language models (PLMs) can be transferred
to various downstream tasks, which exposes a severe security threat. However,
most existing backdoor attacks against PLMs are un-targeted and task-specific.
Few targeted and task-agnostic methods use manually pre-defined triggers and
output representations, which prevent the attacks from being more effective and
general. In this paper, we first summarize the requirements that a more
threatening backdoor attack against PLMs should satisfy, and then propose a new
backdoor attack method called UOR, which breaks the bottleneck of the previous
approach by turning manual selection into automatic optimization. Specifically,
we define poisoned supervised contrastive learning which can automatically
learn the more uniform and universal output representations of triggers for
various PLMs. Moreover, we use gradient search to select appropriate trigger
words which can be adaptive to different PLMs and vocabularies. Experiments
show that our method can achieve better attack performance on various text
classification tasks compared to manual methods. Further, we tested our method
on PLMs with different architectures, different usage paradigms, and more
difficult tasks, which demonstrated the universality of our method.

本论文提出了一种新的后门攻击方法 UOR，通过将手动选择转化为自动优化，定义了毒化监督对比学习，使用梯度搜索选择适当的触发词并针对不同 PLMs 和词汇表进行自适应，取得了比手动方法更好的攻击表现，并证明了该方法的普适性。

UOR：预训练语言模型的通用后门攻击

UOR: Universal Backdoor Attacks on Pre-trained Language Models

Text infilling is defined as a task for filling in the missing part of a
sentence or paragraph, which is suitable for many real-world natural language
generation scenarios. However, given a well-trained sequential generative
model, generating missing symbols conditioned on the context is challenging for
existing greedy approximate inference algorithms. In this paper, we propose an
iterative inference algorithm based on gradient search, which is the first
inference algorithm that can be broadly applied to any neural sequence
generative models for text infilling tasks. We compare the proposed method with
strong baselines on three text infilling tasks with various mask ratios and
different mask strategies. The results show that our proposed method is
effective and efficient for fill-in-the-blank tasks, consistently outperforming
all baselines.

本文提出了一种基于梯度搜索的迭代推断算法，以解决文本填充任务中现有贪心近似推理算法无法应对上下文条件缺失符号生成的挑战，该算法可广泛应用于任何神经序列生成模型。实验结果表明，该算法在不同遮罩比例和遮罩策略的三个文本填充任务中表现出有效和高效的特性，并且始终优于所有基线模型。