Since their inception, Vision Transformers (ViTs) have emerged as a
compelling alternative to Convolutional Neural Networks (CNNs) across a wide
spectrum of tasks. ViTs exhibit notable characteristics, including global
attention, resilience against occlusions, and adaptability to distribution
shifts. One underexplored aspect of ViTs is their potential for multi-attribute
learning, referring to their ability to simultaneously grasp multiple
attribute-related tasks. In this paper, we delve into the multi-attribute
learning capability of ViTs, presenting a straightforward yet effective
strategy for training various attributes through a single ViT network as
distinct tasks. We assess the resilience of multi-attribute ViTs against
adversarial attacks and compare their performance against ViTs designed for
single attributes. Moreover, we further evaluate the robustness of
multi-attribute ViTs against a recent transformer based attack called
Patch-Fool. Our empirical findings on the CelebA dataset provide validation for
our assertion.

通过研究广泛任务中视觉变换器（ViTs）与卷积神经网络（CNNs）的对比，探讨了 ViTs 的多属性学习能力，并提出了通过单个 ViT 网络训练多个属性的简单而有效的策略。通过对 CelebA 数据集的实证研究，验证了多属性 ViTs 在对抗攻击和 Patch-Fool 等最新变换器攻击下的鲁棒性。

多属性视觉转换器的高效稳健学习

Multi-Attribute Vision Transformers are Efficient and Robust Learners

Vision transformers (ViTs) have recently set off a new wave in neural
architecture design thanks to their record-breaking performance in various
vision tasks. In parallel, to fulfill the goal of deploying ViTs into
real-world vision applications, their robustness against potential malicious
attacks has gained increasing attention. In particular, recent works show that
ViTs are more robust against adversarial attacks as compared with convolutional
neural networks (CNNs), and conjecture that this is because ViTs focus more on
capturing global interactions among different input/feature patches, leading to
their improved robustness to local perturbations imposed by adversarial
attacks. In this work, we ask an intriguing question: "Under what kinds of
perturbations do ViTs become more vulnerable learners compared to CNNs?" Driven
by this question, we first conduct a comprehensive experiment regarding the
robustness of both ViTs and CNNs under various existing adversarial attacks to
understand the underlying reason favoring their robustness. Based on the drawn
insights, we then propose a dedicated attack framework, dubbed Patch-Fool, that
fools the self-attention mechanism by attacking its basic component (i.e., a
single patch) with a series of attention-aware optimization techniques.
Interestingly, our Patch-Fool framework shows for the first time that ViTs are
not necessarily more robust than CNNs against adversarial perturbations. In
particular, we find that ViTs are more vulnerable learners compared with CNNs
against our Patch-Fool attack which is consistent across extensive experiments,
and the observations from Sparse/Mild Patch-Fool, two variants of Patch-Fool,
indicate an intriguing insight that the perturbation density and strength on
each patch seem to be the key factors that influence the robustness ranking
between ViTs and CNNs.

本论文探讨了 ViTs 和 CNNs 在面对各种对抗攻击时的鲁棒性及其背后的因素，提出了一种名为 Patch-Fool 的攻击框架，通过对单个 patch 进行一系列 attention-aware 优化技术的攻击来愚弄其 self-attention 机制，并发现在 Patch-Fool 攻击下，ViTs 不一定比 CNNs 更具鲁棒性。