Self-ensemble adversarial training methods improve model robustness by ensembling models at different training epochs, such as model weight averaging (WA). However, previous research has shown that self-ensemble defense methods in adversarial training (AT) still suffer from robust overfitting, which severely affects the generalization performance. Empirically, in the late phases of training, the AT becomes more overfitting to the extent that the individuals for weight averaging also suffer from overfitting and produce anomalous weight values, which causes the self-ensemble model to continue to undergo robust overfitting due to the failure in removing the weight anomalies. To solve this problem, we aim to tackle the influence of outliers in the weight space in this work and propose an easy-to-operate and effective Median-Ensemble Adversarial Training (MEAT) method to solve the robust overfitting phenomenon existing in self-ensemble defense from the source by searching for the median of the historical model weights. Experimental results show that MEAT achieves the best robustness against the powerful AutoAttack and can effectively allievate the robust overfitting. We further demonstrate that most defense methods can improve robust generalization and robustness by combining with MEAT.

自主集成对抗训练方法在不同训练时期通过集成模型，如模型权重平均化（WA），提高模型的健壮性。然而，先前的研究表明，对抗训练中的自主集成防御方法仍然存在健壮过拟合问题，严重影响泛化性能。为解决这个问题，本文目的在于解决权重空间异常值的影响，提出了一种易于操作且有效的中位数集成对抗训练（MEAT）方法，通过搜索历史模型权重的中位数来解决自主集成防御中存在的健壮过拟合现象。实验结果表明，MEAT在强大的AutoAttack攻击下表现出最佳的健壮性，并且能够有效缓解健壮过拟合。我们进一步证明，大多数防御方法与MEAT相结合可以提高健壮泛化性和健壮性。

MEAT: 提高强健性和泛化能力的中值集成对抗训练