We cast doubt on the power of uniform convergence-based generalization bounds to provide a complete picture of why overparameterized deep networks generalize well. While it is well-known that many existing bounds are numerically large, through a variety of experiments, we first bring to light another crucial and more concerning aspect of these bounds: in practice, these bounds can {\em increase} with the dataset size. Guided by our observations, we then show how uniform convergence could provably break down even in a simple setup that preserves the key elements of deep learning: we present a {\em noisy} algorithm that learns a mildly {\em overparameterized} linear classifier such that uniform convergence cannot "explain generalization," even if we take into account implicit regularization {\em to the fullest extent possible}. More precisely, even if we consider only the set of classifiers output by the algorithm that have test errors less than some small $\epsilon$, applying (two-sided) uniform convergence on this set of classifiers yields a generalization guarantee that is larger than $1-\epsilon$ and is therefore nearly vacuous.

通过实验结果，揭示了现存深度学习的多种基于均匀收敛理论的泛化界都是数值较大，因而引起了人们的质疑。而对于使用GD训练的超参数线性分类器和神经网络，即使我们考虑GD的隐式偏差，两边的均匀收敛都无法解释泛化，使得基于均匀收敛的泛化界失去了其解释能力。

均匀收敛可能无法解释深度学习中的泛化