Deep neural networks (DNNs) have been showed to be highly vulnerable to
imperceptible adversarial perturbations. As a complementary type of adversary,
patch attacks that introduce perceptible perturbations to the images have
attracted the interest of researchers. Existing patch attacks rely on the
architecture of the model or the probabilities of predictions and perform
poorly in the decision-based setting, which can still construct a perturbation
with the minimal information exposed -- the top-1 predicted label. In this
work, we first explore the decision-based patch attack. To enhance the attack
efficiency, we model the patches using paired key-points and use targeted
images as the initialization of patches, and parameter optimizations are all
performed on the integer domain. Then, we propose a differential evolutionary
algorithm named DevoPatch for query-efficient decision-based patch attacks.
Experiments demonstrate that DevoPatch outperforms the state-of-the-art
black-box patch attacks in terms of patch area and attack success rate within a
given query budget on image classification and face verification. Additionally,
we conduct the vulnerability evaluation of ViT and MLP on image classification
in the decision-based patch attack setting for the first time. Using DevoPatch,
we can evaluate the robustness of models to black-box patch attacks. We believe
this method could inspire the design and deployment of robust vision models
based on various DNN architectures in the future.

本文提出了一种增强的基于对抗样本攻击方式，并探索了不依赖于网络输出数量的决策性基于修补程序的攻击方法，使用称为 DevoPatch 的差分进化算法。实验证明 DevoPatch 优于现有方法，在给定查询预算内攻击成功率更高。