Self-supervised learning in computer vision trains on unlabeled data, such as
images or (image, text) pairs, to obtain an image encoder that learns
high-quality embeddings for input data. Emerging backdoor attacks towards
encoders expose crucial vulnerabilities of self-supervised learning, since
downstream classifiers (even further trained on clean data) may inherit
backdoor behaviors from encoders. Existing backdoor detection methods mainly
focus on supervised learning settings and cannot handle pre-trained encoders
especially when input labels are not available. In this paper, we propose
DECREE, the first backdoor detection approach for pre-trained encoders,
requiring neither classifier headers nor input labels. We evaluate DECREE on
over 400 encoders trojaned under 3 paradigms. We show the effectiveness of our
method on image encoders pre-trained on ImageNet and OpenAI's CLIP 400 million
image-text pairs. Our method consistently has a high detection accuracy even if
we have only limited or no access to the pre-training dataset.

本文提出了 DECREE，是第一个用于预训练编码器的后门检测方法，无需分类器头或输入标签，并通过对 400 多个遭受三种范式攻击的编码器的评估，在 ImageNet 和 CLIP 400 百万图像文本对上，我们的方法一致具有很高的检测准确性，即使我们只有有限或没有访问预训练数据集。