We study the implicit bias of optimization in robust empirical risk
minimization (robust ERM) and its connection with robust generalization. In
classification settings under adversarial perturbations with linear models, we
study what type of regularization should ideally be applied for a given
perturbation set to improve (robust) generalization. We then show that the
implicit bias of optimization in robust ERM can significantly affect the
robustness of the model and identify two ways this can happen; either through
the optimization algorithm or the architecture. We verify our predictions in
simulations with synthetic data and experimentally study the importance of
implicit bias in robust ERM with deep neural networks.

我们研究了鲁棒经验风险最小化（鲁棒 ERM）中的优化隐性偏差及其与鲁棒泛化的关系。在面对带有线性模型的对抗干扰的分类设置下，我们研究了应该为给定的扰动集合理想地应用什么类型的正则化来改善（鲁棒）泛化。然后，我们表明鲁棒 ERM 中的优化隐性偏差可以显著影响模型的鲁棒性，并确定了两种方式：通过优化算法或架构。我们通过合成数据的模拟验证了我们的预测，并实验性地研究了深度神经网络中优化隐性偏差在鲁棒 ERM 中的重要性。

对抗鲁棒广义化中隐性偏见的代价

The Price of Implicit Bias in Adversarially Robust Generalization

We consider robust empirical risk minimization (ERM), where model parameters
are chosen to minimize the worst-case empirical loss when each data point
varies over a given convex uncertainty set. In some simple cases, such problems
can be expressed in an analytical form. In general the problem can be made
tractable via dualization, which turns a min-max problem into a min-min
problem. Dualization requires expertise and is tedious and error-prone. We
demonstrate how CVXPY can be used to automate this dualization procedure in a
user-friendly manner. Our framework allows practitioners to specify and solve
robust ERM problems with a general class of convex losses, capturing many
standard regression and classification problems. Users can easily specify any
complex uncertainty set that is representable via disciplined convex
programming (DCP) constraints.

本文通过使用 CVXPY 自动执行内部的二次规划过程，演示了如何利用凸不确定集来解决一般的分类和回归问题以及其他一些常见的损失函数的确定性限制。