We introduce the concept of deceptive diffusion -- training a generative AI
model to produce adversarial images. Whereas a traditional adversarial attack
algorithm aims to perturb an existing image to induce a misclassificaton, the
deceptive diffusion model can create an arbitrary number of new, misclassified
images that are not directly associated with training or test images. Deceptive
diffusion offers the possibility of strengthening defence algorithms by
providing adversarial training data at scale, including types of
misclassification that are otherwise difficult to find. In our experiments, we
also investigate the effect of training on a partially attacked data set. This
highlights a new type of vulnerability for generative diffusion models: if an
attacker is able to stealthily poison a portion of the training data, then the
resulting diffusion model will generate a similar proportion of misleading
outputs.

引入了欺骗扩散的概念 -- 训练生成 AI 模型以产生具有对抗性的图像。与传统的对抗性攻击算法不同，欺骗扩散模型可以创建任意数量的新的、被错误分类的图像，这些图像与训练或测试图像没有直接关联。欺骗扩散在规模上提供了对抗性训练数据以加强防御算法，包括在其他情况下难以找到的错误分类类型。在我们的实验中，我们还研究了在部分受攻击数据集上进行训练的效果。这突出了生成扩散模型的一种新型漏洞：如果攻击者能够秘密地污染部分训练数据，那么生成的扩散模型将产生相似比例的误导输出。