Adversarially trained models exhibit a large generalization gap: they can
interpolate the training set even for large perturbation radii, but at the cost
of large test error on clean samples. To investigate this gap, we decompose the
test risk into its bias and variance components and study their behavior as a
function of adversarial training perturbation radii ($\varepsilon$). We find
that the bias increases monotonically with $\varepsilon$ and is the dominant
term in the risk. Meanwhile, the variance is unimodal as a function of
$\varepsilon$, peaking near the interpolation threshold for the training set.
This characteristic behavior occurs robustly across different datasets and also
for other robust training procedures such as randomized smoothing. It thus
provides a test for proposed explanations of the generalization gap. We find
that some existing explanations fail this test--for instance, by predicting a
monotonically increasing variance curve. This underscores the power of
bias-variance decompositions in modern settings-by providing two measurements
instead of one, they can rule out more explanations than test accuracy alone.
We also show that bias and variance can provide useful guidance for scalably
reducing the generalization gap, highlighting pre-training and unlabeled data
as promising routes.

通过偏差方差分解研究对抗性训练扰动半径对模型测试误差的影响，发现模型的偏差随扰动半径增加单调增加，方差则在训练集插值阈值附近单峰；同时，偏差和方差可用于指导缩小泛化间隔的方法，即预训练和使用无标签数据。