Deep neural networks (DNNs) are proved to be vulnerable against backdoor
attacks. A backdoor is often embedded in the target DNNs through injecting a
backdoor trigger into training examples, which can cause the target DNNs
misclassify an input attached with the backdoor trigger. Existing backdoor
detection methods often require the access to the original poisoned training
data, the parameters of the target DNNs, or the predictive confidence for each
given input, which are impractical in many real-world applications, e.g.,
on-device deployed DNNs. We address the black-box hard-label backdoor detection
problem where the DNN is fully black-box and only its final output label is
accessible. We approach this problem from the optimization perspective and show
that the objective of backdoor detection is bounded by an adversarial
objective. Further theoretical and empirical studies reveal that this
adversarial objective leads to a solution with highly skewed distribution; a
singularity is often observed in the adversarial map of a backdoor-infected
example, which we call the adversarial singularity phenomenon. Based on this
observation, we propose the adversarial extreme value analysis(AEVA) to detect
backdoors in black-box neural networks. AEVA is based on an extreme value
analysis of the adversarial map, computed from the monte-carlo gradient
estimation. Evidenced by extensive experiments across multiple popular tasks
and backdoor attacks, our approach is shown effective in detecting backdoor
attacks under the black-box hard-label scenarios.

研究针对黑盒深度神经网络的后门检测问题，提出一种基于极值分析的方法（AEVA），该方法可在只有最终输出标签可访问时检测黑客攻击的后门。

AEVA：使用对抗极值分析进行黑盒后门检测

AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value  Analysis

Although deep neural networks (DNNs) have made rapid progress in recent
years, they are vulnerable in adversarial environments. A malicious backdoor
could be embedded in a model by poisoning the training dataset, whose intention
is to make the infected model give wrong predictions during inference when the
specific trigger appears. To mitigate the potential threats of backdoor
attacks, various backdoor detection and defense methods have been proposed.
However, the existing techniques usually require the poisoned training data or
access to the white-box model, which is commonly unavailable in practice. In
this paper, we propose a black-box backdoor detection (B3D) method to identify
backdoor attacks with only query access to the model. We introduce a
gradient-free optimization algorithm to reverse-engineer the potential trigger
for each class, which helps to reveal the existence of backdoor attacks. In
addition to backdoor detection, we also propose a simple strategy for reliable
predictions using the identified backdoored models. Extensive experiments on
hundreds of DNN models trained on several datasets corroborate the
effectiveness of our method under the black-box setting against various
backdoor attacks.

该论文提出一种基于黑盒模型的反向工程优化算法，用于检测深度神经网络中嵌入的恶意后门攻击，并通过检测结果进行有效的可靠预测，实验表明其可以有效地应对多种后门攻击。