In recent years, Deep Neural Network models have been developed in different
fields, where they have brought many advances. However, they have also started
to be used in tasks where risk is critical. A misdiagnosis of these models can
lead to serious accidents or even death. This concern has led to an interest
among researchers to study possible attacks on these models, discovering a long
list of vulnerabilities, from which every model should be defended. The
adversarial example attack is a widely known attack among researchers, who have
developed several defenses to avoid such a threat. However, these defenses are
as opaque as a deep neural network model, how they work is still unknown. This
is why visualizing how they change the behavior of the target model is
interesting in order to understand more precisely how the performance of the
defended model is being modified. For this work, some defenses, against
adversarial example attack, have been selected in order to visualize the
behavior modification of each of them in the defended model. Adversarial
training, dimensionality reduction and prediction similarity were the selected
defenses, which have been developed using a model composed by convolution
neural network layers and dense neural network layers. In each defense, the
behavior of the original model has been compared with the behavior of the
defended model, representing the target model by a graph in a visualization.

近年来，深度神经网络模型在各个领域取得了许多进展，但是它们也开始在风险关键任务中使用。模型的误诊可能导致严重事故甚至死亡。这引起了研究人员对这些模型的攻击进行研究，发现了一长串的漏洞，所以每个模型都应该进行防护。对抗性示例攻击是研究人员中广为人知的一种攻击，他们已经开发了一些防御措施来避免这种威胁。然而，这些防御措施与深度神经网络模型一样晦涩难懂，其工作原理仍然未知。因此，通过可视化它们如何改变目标模型的行为对于更精确地理解防御模型的性能修改是有趣的。在这项工作中，选择了一些对抗性示例攻击的防御措施，并以可视化的方式显示了每个防御措施在防御模型中的行为变化。选择了对抗训练、降维和预测相似度作为防御措施，使用了由卷积神经网络层和全连接神经网络层组成的模型。在每个防御方案中，将原始模型的行为与防御模型的行为进行对比，并通过可视化图表来表示目标模型。

通过可视化动态风险评估理解深度学习对抗性示例的防御

Understanding Deep Learning defenses Against Adversarial Examples  Through Visualizations for Dynamic Risk Assessment

This paper demonstrates that Non-Maximum Suppression (NMS), which is commonly
used in Object Detection (OD) tasks to filter redundant detection results, is
no longer secure. Considering that NMS has been an integral part of OD systems,
thwarting the functionality of NMS can result in unexpected or even lethal
consequences for such systems. In this paper, an adversarial example attack
which triggers malfunctioning of NMS in end-to-end OD models is proposed. The
attack, namely \texttt{Daedalus}, compresses the dimensions of detection boxes
to evade NMS. As a result, the final detection output contains extremely dense
false positives. This can be fatal for many OD applications such as autonomous
vehicles and surveillance systems. The attack can be generalised to different
end-to-end OD models, such that the attack cripples various OD applications.
Furthermore, a way to craft robust adversarial examples is developed by using
an ensemble of popular detection models as the substitutes. Considering the
pervasive nature of model reusing in real-world OD scenarios, Daedalus examples
crafted based on an \textit{ensemble of substitutes} can launch attacks without
knowing the parameters of the victim models. Experimental results demonstrate
that the attack effectively stops NMS from filtering redundant bounding boxes.
As the evaluation results suggest, Daedalus increases the false positive rate
in detection results to $99.9\%$ and reduces the mean average precision scores
to $0$, while maintaining a low cost of distortion on the original inputs. It
is also demonstrated that the attack can be practically launched against
real-world OD systems via printed posters.

本文提出了一种名为 Daedalus 的对抗性样本攻击，通过缩小检测框的尺寸来规避 Non-Maximum Suppression，为现实中的物体检测系统带来潜在的致命后果。该攻击能够以低代价对原始输入进行干扰，同时可用于印刷海报等现实场景中对物体检测系统的攻击。

Daedalus：通过对抗样本打破目标检测中的非最大抑制

Daedalus: Breaking Non-Maximum Suppression in Object Detection via  Adversarial Examples

Machine learning systems based on deep neural networks, being able to produce
state-of-the-art results on various perception tasks, have gained mainstream
adoption in many applications. However, they are shown to be vulnerable to
adversarial example attack, which generates malicious output by adding slight
perturbations to the input. Previous adversarial example crafting methods,
however, use simple metrics to evaluate the distances between the original
examples and the adversarial ones, which could be easily detected by human
eyes. In addition, these attacks are often not robust due to the inevitable
noises and deviation in the physical world. In this work, we present a new
adversarial example attack crafting method, which takes the human perceptual
system into consideration and maximizes the noise tolerance of the crafted
adversarial example. Experimental results demonstrate the efficacy of the
proposed technique.

该研究提出了一种新的对抗样本攻击方法，考虑到人类感知系统并最大化制作的对抗样本的噪声容忍度，实验结果证明了该技术的有效性。