Deep neural networks are susceptible to adversarial attacks, which can compromise their performance and accuracy. Adversarial Training (AT) has emerged as a popular approach for protecting neural networks against such attacks. However, a key challenge of AT is robust overfitting, where the network's robust performance on test data deteriorates with further training, thus hindering generalization. Motivated by the concept of active forgetting in the brain, we introduce a novel learning paradigm called "Forget to Mitigate Overfitting (FOMO)". FOMO alternates between the forgetting phase, which randomly forgets a subset of weights and regulates the model's information through weight reinitialization, and the relearning phase, which emphasizes learning generalizable features. Our experiments on benchmark datasets and adversarial attacks show that FOMO alleviates robust overfitting by significantly reducing the gap between the best and last robust test accuracy while improving the state-of-the-art robustness. Furthermore, FOMO provides a better trade-off between standard and robust accuracy, outperforming baseline adversarial methods. Finally, our framework is robust to AutoAttacks and increases generalization in many real-world scenarios.

通过引入一种名为“忘却以减轻过拟合（FOMO）”的新型学习范式，交替进行随机遗忘权重子集和重新初始化权重的调节，以及强调学习可泛化特征的学习阶段，FOMO成功缓解了强健过拟合问题，显著减小了最佳和最后的强健测试准确率之间的差距，并提高了最先进的鲁棒性，在标准准确性和鲁棒准确性之间提供了更好的权衡正。此外，FOMO对AutoAttacks具有鲁棒性，并在许多实际情境中提高了泛化能力。

随机遗忘对强健泛化的有效性