Adversarial training augments the training set with perturbations to improve
the robust error (over worst-case perturbations), but it often leads to an
increase in the standard error (on unperturbed test inputs). Previous
explanations for this tradeoff rely on the assumption that no predictor in the
hypothesis class has low standard and robust error. In this work, we precisely
characterize the effect of augmentation on the standard error in linear
regression when the optimal linear predictor has zero standard and robust
error. In particular, we show that the standard error could increase even when
the augmented perturbations have noiseless observations from the optimal linear
predictor. We then prove that the recently proposed robust self-training (RST)
estimator improves robust error without sacrificing standard error for
noiseless linear regression. Empirically, for neural networks, we find that RST
with different adversarial training methods improves both standard and robust
error for random and adversarial rotations and adversarial $\ell_\infty$
perturbations in CIFAR-10.

本研究通过对线性回归中对最优线性预测器进行噪声扰动的方法，准确描述了数据增广对于标准误差的影响，并证明了最近提出的鲁棒自学习估计器（RST）可以在不损失标准误差的情况下提高鲁棒误差。经验证，使用不同的对抗训练方法进行 RST 可以改善 CIFAR-10 数据集上的标准误差和鲁棒误差。

理解和减轻稳健性与准确性之间的权衡

Understanding and Mitigating the Tradeoff Between Robustness and  Accuracy

While adversarial training can improve robust accuracy (against an
adversary), it sometimes hurts standard accuracy (when there is no adversary).
Previous work has studied this tradeoff between standard and robust accuracy,
but only in the setting where no predictor performs well on both objectives in
the infinite data limit. In this paper, we show that even when the optimal
predictor with infinite data performs well on both objectives, a tradeoff can
still manifest itself with finite data. Furthermore, since our construction is
based on a convex learning problem, we rule out optimization concerns, thus
laying bare a fundamental tension between robustness and generalization.
Finally, we show that robust self-training mostly eliminates this tradeoff by
leveraging unlabeled data.

本文研究了对抗训练在提高鲁棒精度（对抗方面）的同时又有可能降低标准精度（没有对抗方面）。通过构造凸学习问题，我们发现鲁棒精度和泛化能力之间存在基本的紧张关系，而利用未标记的数据进行鲁棒自我训练可以消除这种关系。