Adversarial training is a promising method to improve the robustness against
adversarial attacks. To enhance its performance, recent methods impose high
weights on the cross-entropy loss for important data points near the decision
boundary. However, these importance-aware methods are vulnerable to
sophisticated attacks, e.g., Auto-Attack. In this paper, we experimentally
investigate the cause of their vulnerability via margins between logits for the
true label and the other labels because they should be large enough to prevent
the largest logit from being flipped by the attacks. Our experiments reveal
that the histogram of the logit margins of naïve adversarial training has two
peaks. Thus, the levels of difficulty in increasing logit margins are roughly
divided into two: difficult samples (small logit margins) and easy samples
(large logit margins). On the other hand, only one peak near zero appears in
the histogram of importance-aware methods, i.e., they reduce the logit margins
of easy samples. To increase logit margins of difficult samples without
reducing those of easy samples, we propose switching one-versus-the-rest loss
(SOVR), which switches from cross-entropy to one-versus-the-rest loss (OVR) for
difficult samples. We derive trajectories of logit margins for a simple problem
and prove that OVR increases logit margins two times larger than the weighted
cross-entropy loss. Thus, SOVR increases logit margins of difficult samples,
unlike existing methods. We experimentally show that SOVR achieves better
robustness against Auto-Attack than importance-aware methods.

本文通过对 logit margins 的研究，提出了 SOVR 方法来提高 Adversarial training 的威胁鲁棒性并且在实验中展示了其效果。