This paper introduces new parameterizations of equilibrium neural networks,
i.e. networks defined by implicit equations. This model class includes standard
multilayer and residual networks as special cases. The new parameterization
admits a Lipschitz bound during training via unconstrained optimization: no
projections or barrier functions are required. Lipschitz bounds are a common
proxy for robustness and appear in many generalization bounds. Furthermore,
compared to previous works we show well-posedness (existence of solutions)
under less restrictive conditions on the network weights and more natural
assumptions on the activation functions: that they are monotone and slope
restricted. These results are proved by establishing novel connections with
convex optimization, operator splitting on non-Euclidean spaces, and
contracting neural ODEs. In image classification experiments we show that the
Lipschitz bounds are very accurate and improve robustness to adversarial
attacks.

本研究提出了新的平衡神经网络参数化方法，该方法可以实现在训练期间的 Lipschitz bound 并提升强健性，并通过建立与凸优化、非欧几里得空间上的算子分裂和收缩神经微分方程的新连接来证明这些结果，在图像分类实验中表现出非常高的准确性和抵御对抗性攻击的能力。