We establish a margin based data dependent generalization error bound for a
general family of deep neural networks in terms of the depth and width, as well
as the Jacobian of the networks. Through introducing a new characterization of
the Lipschitz properties of neural network family, we achieve significantly
tighter generalization bounds than existing results. Moreover, we show that the
generalization bound can be further improved for bounded losses. Aside from the
general feedforward deep neural networks, our results can be applied to derive
new bounds for popular architectures, including convolutional neural networks
(CNNs) and residual networks (ResNets). When achieving same generalization
errors with previous arts, our bounds allow for the choice of larger parameter
spaces of weight matrices, inducing potentially stronger expressive ability for
neural networks. Numerical evaluation is also provided to support our theory.

我们通过引入一种新的神经网络族的 Lipschitz 属性的表征，建立了一种基于边际的数据相关的广泛深度神经网络泛化误差界限，将其与深度、宽度以及网络的雅可比矩阵联系起来。和现有的结果相比，我们实现了明显更紧的泛化下界，并且还表明，对于损失有界的情况，可以进一步改善泛化下界。除了一般的前馈深度神经网络外，我们的结果可以应用于推导流行的体系结构，包括卷积神经网络和残差网络。在实现与上一项艺术相同的泛化误差的同时，我们的误差允许选择更大的权重矩阵参数空间，从而为神经网络提供更强的表达能力。其中还提供了数值评估来支持我们的理论。

深度神经网络的更紧密的泛化界限：卷积神经网络、ResNet 等

On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets,  and Beyond

In this paper we discuss the stability properties of convolutional neural
networks. Convolutional neural networks are widely used in machine learning. In
classification they are mainly used as feature extractors. Ideally, we expect
similar features when the inputs are from the same class. That is, we hope to
see a small change in the feature vector with respect to a deformation on the
input signal. This can be established mathematically, and the key step is to
derive the Lipschitz properties. Further, we establish that the stability
results can be extended for more general networks. We give a formula for
computing the Lipschitz bound, and compare it with other methods to show it is
closer to the optimal value.

本文讨论卷积神经网络的稳定性，以及基于 Lipschitz 性质的特征提取和分类方法，计算了 Lipschitz 边界并将其值与其他方法进行了比较，验证了 Lipschitz 边界计算方法的优越性。