Universal adversarial perturbations (UAPs), a.k.a. input-agnostic
perturbations, has been proved to exist and be able to fool cutting-edge deep
learning models on most of the data samples. Existing UAP methods mainly focus
on attacking image classification models. Nevertheless, little attention has
been paid to attacking image retrieval systems. In this paper, we make the
first attempt in attacking image retrieval systems. Concretely, image retrieval
attack is to make the retrieval system return irrelevant images to the query at
the top ranking list. It plays an important role to corrupt the neighbourhood
relationships among features in image retrieval attack. To this end, we propose
a novel method to generate retrieval-against UAP to break the neighbourhood
relationships of image features via degrading the corresponding ranking metric.
To expand the attack method to scenarios with varying input sizes or
untouchable network parameters, a multi-scale random resizing scheme and a
ranking distillation strategy are proposed. We evaluate the proposed method on
four widely-used image retrieval datasets, and report a significant performance
drop in terms of different metrics, such as mAP and mP@10. Finally, we test our
attack methods on the real-world visual search engine, i.e., Google Images,
which demonstrates the practical potentials of our methods.

首次尝试以攻击图像检索系统为目标，利用新的方法生成反检索通用对抗扰动，通过降低相关的排名度量来打破图像特征之间的邻域关系。我们的方法通过多尺度随机调整大小的方案和排名蒸馏策略扩展攻击方法，并在四个广泛使用的图像检索数据集上进行了评估。在不同度量方面，如 mAP 和 mP @ 10 等，报告了显着的性能下降。最后，我们在实际的视觉搜索引擎 Google Images 上测试了我们的攻击方法，证明了我们的方法的实际潜力。