Denoising probabilistic diffusion models have shown breakthrough performance
that can generate more photo-realistic images or human-level illustrations than
the prior models such as GANs. This high image-generation capability has
stimulated the creation of many downstream applications in various areas.
However, we find that this technology is indeed a double-edged sword: We
identify a new type of attack, called the Natural Denoising Diffusion (NDD)
attack based on the finding that state-of-the-art deep neural network (DNN)
models still hold their prediction even if we intentionally remove their robust
features, which are essential to the human visual system (HVS), by text
prompts. The NDD attack can generate low-cost, model-agnostic, and
transferrable adversarial attacks by exploiting the natural attack capability
in diffusion models. Motivated by the finding, we construct a large-scale
dataset, Natural Denoising Diffusion Attack (NDDA) dataset, to systematically
evaluate the risk of the natural attack capability of diffusion models with
state-of-the-art text-to-image diffusion models. We evaluate the natural attack
capability by answering 6 research questions. Through a user study to confirm
the validity of the NDD attack, we find that the NDD attack can achieve an 88%
detection rate while being stealthy to 93% of human subjects. We also find that
the non-robust features embedded by diffusion models contribute to the natural
attack capability. To confirm the model-agnostic and transferrable attack
capability, we perform the NDD attack against an AD vehicle and find that 73%
of the physically printed attacks can be detected as a stop sign. We hope that
our study and dataset can help our community to be aware of the risk of
diffusion models and facilitate further research toward robust DNN models.

通过发现深度神经网络模型仍能保持预测能力，即使有意删除其对人类视觉系统至关重要的稳健特征，我们识别出了一种名为 “自然去噪扩散攻击（NDD）” 的新型攻击。该攻击能够以低成本并且与模型无关、可转换为对抗性攻击，并利用扩散模型中的自然攻击能力。我们构建了一个大规模数据集 “自然去噪扩散攻击（NDDA）数据集”，来系统评估最新的文本 - 图像扩散模型的自然攻击能力的风险。我们通过回答 6 个研究问题来评估自然攻击能力，并通过用户研究证实了 NDD 攻击的有效性，发现 NDD 攻击能够实现 88% 的检测率，同时对 93% 的被试者具有隐蔽性。我们还发现，扩散模型嵌入的非稳健特征对于自然攻击能力起到了贡献作用。为了证明攻击的模型无关性和可转换性，我们对一辆自动驾驶汽车进行了 NDD 攻击，发现 73% 的物理打印的攻击能够被识别为停止标志。希望我们的研究和数据集能够帮助社区意识到扩散模型的风险，并促进更多关于强大深度神经网络模型的研究。