Deep neural networks obtained by standard training have been constantly
plagued by adversarial examples. Although adversarial training demonstrates its
capability to defend against adversarial examples, unfortunately, it leads to
an inevitable drop in the natural generalization. To address the issue, we
decouple the natural generalization and the robust generalization from joint
training and formulate different training strategies for each one.
Specifically, instead of minimizing a global loss on the expectation over these
two generalization errors, we propose a bi-expert framework called
\emph{Generalist} where we simultaneously train base learners with task-aware
strategies so that they can specialize in their own fields. The parameters of
base learners are collected and combined to form a global learner at intervals
during the training process. The global learner is then distributed to the base
learners as initialized parameters for continued training. Theoretically, we
prove that the risks of Generalist will get lower once the base learners are
well trained. Extensive experiments verify the applicability of Generalist to
achieve high accuracy on natural examples while maintaining considerable
robustness to adversarial ones. Code is available at
this https URL.

通过提出基于任务感知的双专家框架 Generalist，将自然泛化和鲁棒泛化分离，为每个领域专业基础学习器选择不同的训练策略，并将它们的参数收集并组合成全局学习器进行训练，最终在自然表现和对抗性表现方面均能获得良好的结果。

Generalist: 自然与强健泛化的解耦

Generalist: Decoupling Natural and Robust Generalization

Adversarial training based on the minimax formulation is necessary for
obtaining adversarial robustness of trained models. However, it is conservative
or even pessimistic so that it sometimes hurts the natural generalization. In
this paper, we raise a fundamental question---do we have to trade off natural
generalization for adversarial robustness? We argue that adversarial training
is to employ confident adversarial data for updating the current model. We
propose a novel approach of friendly adversarial training (FAT): rather than
employing most adversarial data maximizing the loss, we search for least
adversarial (i.e., friendly adversarial) data minimizing the loss, among the
adversarial data that are confidently misclassified. Our novel formulation is
easy to implement by just stopping the most adversarial data searching
algorithms such as PGD (projected gradient descent) early, which we call
early-stopped PGD. Theoretically, FAT is justified by an upper bound of the
adversarial risk. Empirically, early-stopped PGD allows us to answer the
earlier question negatively---adversarial robustness can indeed be achieved
without compromising the natural generalization.

本研究提出了友好对抗训练 (FAT) 的新方法，其中通过提前停止最严格的对抗数据搜索算法，即早停止的 PGD，来最小化损失并利用自信的对抗数据更新当前模型，理论上可以通过对抗风险的上限来证明，实验证明不需要以自然泛化为代价也可以实现对抗强度。