Deep neural networks have achieved state-of-the-art performance on various
tasks. However, lack of interpretability and transparency makes it easier for
malicious attackers to inject trojan backdoor into the neural networks, which
will make the model behave abnormally when a backdoor sample with a specific
trigger is input. In this paper, we propose NeuronInspect, a framework to
detect trojan backdoors in deep neural networks via output explanation
techniques. NeuronInspect first identifies the existence of backdoor attack
targets by generating the explanation heatmap of the output layer. We observe
that generated heatmaps from clean and backdoored models have different
characteristics. Therefore we extract features that measure the attributes of
explanations from an attacked model namely: sparse, smooth and persistent. We
combine these features and use outlier detection to figure out the outliers,
which is the set of attack targets. We demonstrate the effectiveness and
efficiency of NeuronInspect on MNIST digit recognition dataset and GTSRB
traffic sign recognition dataset. We extensively evaluate NeuronInspect on
different attack scenarios and prove better robustness and effectiveness over
state-of-the-art trojan backdoor detection techniques Neural Cleanse by a great
margin.

本文提出了一种名为 NeuronInspect 的深度神经网络检测框架，可通过输出解释技术检测深度神经网络中的 trojan backdoors，并应用于 MNIST 数字识别和 GTSRB 交通标志识别数据集中，通过异常检测和特征提取来获得更好的鲁棒性和准确性