Numerous generalization bounds have been proposed in the literature as
potential explanations for the ability of neural networks to generalize in the
overparameterized setting. However, none of these bounds are tight. For
instance, in their paper ``Fantastic Generalization Measures and Where to Find
Them'', Jiang et al. (2020) examine more than a dozen generalization bounds,
and show empirically that none of them imply guarantees that can explain the
remarkable performance of neural networks. This raises the question of whether
tight generalization bounds are at all possible. We consider two types of
generalization bounds common in the literature: (1) bounds that depend on the
training set and the output of the learning algorithm. There are multiple
bounds of this type in the literature (e.g., norm-based and margin-based
bounds), but we prove mathematically that no such bound can be uniformly tight
in the overparameterized setting; (2) bounds that depend on the training set
and on the learning algorithm (e.g., stability bounds). For these bounds, we
show a trade-off between the algorithm's performance and the bound's tightness.
Namely, if the algorithm achieves good accuracy on certain distributions in the
overparameterized setting, then no generalization bound can be tight for it. We
conclude that generalization bounds in the overparameterized setting cannot be
tight without suitable assumptions on the population distribution.

神经网络在超参数化设置中的泛化界限无法达到紧致性，除非在总体分布上具备合适的假设。