We propose a general framework for deriving generalization bounds for parallel positively homogeneous neural networks--a class of neural networks whose input-output map decomposes as the sum of positively homogeneous maps. Examples of such networks include matrix factorization and sensing, single-layer multi-head attention mechanisms, tensor factorization, deep linear and ReLU networks, and more. Our general framework is based on linking the non-convex empirical risk minimization (ERM) problem to a closely related convex optimization problem over prediction functions, which provides a global, achievable lower-bound to the ERM problem. We exploit this convex lower-bound to perform generalization analysis in the convex space while controlling the discrepancy between the convex model and its non-convex counterpart. We apply our general framework to a wide variety of models ranging from low-rank matrix sensing, to structured matrix sensing, two-layer linear networks, two-layer ReLU networks, and single-layer multi-head attention mechanisms, achieving generalization bounds with a sample complexity that scales almost linearly with the network width.

本研究解决了并行正齐次神经网络的泛化界限推导问题，提出了一种新的框架，通过将非凸经验风险最小化问题与相关的凸优化问题联系起来，提供了可实现的全局下界。该框架的应用使得在多种模型中实现了样本复杂度与网络宽度几乎线性缩放的泛化界限，具有重要的理论和实际意义。

一种凸放松方法用于并行正齐次网络的泛化分析