Self-supervised learning (SSL) is increasingly attractive for pre-training
encoders without requiring labeled data. Downstream tasks built on top of those
pre-trained encoders can achieve nearly state-of-the-art performance. The
pre-trained encoders by SSL, however, are vulnerable to backdoor attacks as
demonstrated by existing studies. Numerous backdoor mitigation techniques are
designed for downstream task models. However, their effectiveness is impaired
and limited when adapted to pre-trained encoders, due to the lack of label
information when pre-training. To address backdoor attacks against pre-trained
encoders, in this paper, we innovatively propose a mutual information guided
backdoor mitigation technique, named MIMIC. MIMIC treats the potentially
backdoored encoder as the teacher net and employs knowledge distillation to
distill a clean student encoder from the teacher net. Different from existing
knowledge distillation approaches, MIMIC initializes the student with random
weights, inheriting no backdoors from teacher nets. Then MIMIC leverages mutual
information between each layer and extracted features to locate where benign
knowledge lies in the teacher net, with which distillation is deployed to clone
clean features from teacher to student. We craft the distillation loss with two
aspects, including clone loss and attention loss, aiming to mitigate backdoors
and maintain encoder performance at the same time. Our evaluation conducted on
two backdoor attacks in SSL demonstrates that MIMIC can significantly reduce
the attack success rate by only utilizing <5% of clean data, surpassing seven
state-of-the-art backdoor mitigation techniques.

本文提出了一种名为 MIMIC 的互信息引导的后门缓解技术，通过知识蒸馏从潜在后门编码器中提取干净的学生编码器，以减轻后门攻击并同时保持编码器性能。在自我监督学习中对两个后门攻击进行的评估表明，MIMIC 可以显著降低攻击成功率，仅利用不到 5% 的干净数据，超过七种最先进的后门缓解技术。

基于互信息引导的事先训练编码器后门缓解

Mutual Information Guided Backdoor Mitigation for Pre-trained Encoders

In this demo paper, we present the XFake system, an explainable fake news
detector that assists end-users to identify news credibility. To effectively
detect and interpret the fakeness of news items, we jointly consider both
attributes (e.g., speaker) and statements. Specifically, MIMIC, ATTN and PERT
frameworks are designed, where MIMIC is built for attribute analysis, ATTN is
for statement semantic analysis and PERT is for statement linguistic analysis.
Beyond the explanations extracted from the designed frameworks, relevant
supporting examples as well as visualization are further provided to facilitate
the interpretation. Our implemented system is demonstrated on a real-world
dataset crawled from PolitiFact, where thousands of verified political news
have been collected.

XFake 是一个可解释的 fake news detector，通过同时考虑发言特征和言论语义、语言分析，设计了 MIMIC、ATTN 和 PERT 框架，在提供相关支持示例和可视化的同时，帮助人们正确辨别真假新闻。