Adversarial training augments the training set with perturbations to improve
the robust error (over worst-case perturbations), but it often leads to an
increase in the standard error (on unperturbed test inputs). Previous
explanations for this tradeoff rely on the assumption that no predictor in the
hypothesis class has low standard and robust error. In this work, we precisely
characterize the effect of augmentation on the standard error in linear
regression when the optimal linear predictor has zero standard and robust
error. In particular, we show that the standard error could increase even when
the augmented perturbations have noiseless observations from the optimal linear
predictor. We then prove that the recently proposed robust self-training (RST)
estimator improves robust error without sacrificing standard error for
noiseless linear regression. Empirically, for neural networks, we find that RST
with different adversarial training methods improves both standard and robust
error for random and adversarial rotations and adversarial $\ell_\infty$
perturbations in CIFAR-10.

本研究通过对线性回归中对最优线性预测器进行噪声扰动的方法，准确描述了数据增广对于标准误差的影响，并证明了最近提出的鲁棒自学习估计器（RST）可以在不损失标准误差的情况下提高鲁棒误差。经验证，使用不同的对抗训练方法进行 RST 可以改善 CIFAR-10 数据集上的标准误差和鲁棒误差。