We study the multiple manifold problem, a binary classification task modeled
on applications in machine vision, in which a deep fully-connected neural
network is trained to separate two low-dimensional submanifolds of the unit
sphere. We provide an analysis of the one-dimensional case, proving for a
simple manifold configuration that when the network depth $L$ is large relative
to certain geometric and statistical properties of the data, the network width
$n$ grows as a sufficiently large polynomial in $L$, and the number of i.i.d.
samples from the manifolds is polynomial in $L$, randomly-initialized gradient
descent rapidly learns to classify the two manifolds perfectly with high
probability. Our analysis demonstrates concrete benefits of depth and width in
the context of a practically-motivated model problem: the depth acts as a
fitting resource, with larger depths corresponding to smoother networks that
can more readily separate the class manifolds, and the width acts as a
statistical resource, enabling concentration of the randomly-initialized
network and its gradients. The argument centers around the neural tangent
kernel and its role in the nonasymptotic analysis of training overparameterized
neural networks; to this literature, we contribute essentially optimal rates of
concentration for the neural tangent kernel of deep fully-connected networks,
requiring width $n \gtrsim L\,\mathrm{poly}(d_0)$ to achieve uniform
concentration of the initial kernel over a $d_0$-dimensional submanifold of the
unit sphere $\mathbb{S}^{n_0-1}$, and a nonasymptotic framework for
establishing generalization of networks trained in the NTK regime with
structured data. The proof makes heavy use of martingale concentration to
optimally treat statistical dependencies across layers of the initial random
network. This approach should be of use in establishing similar results for
other network architectures.

研究神经网络中的多重流形问题，证明当网络深度相对于数据的几何和统计属性较大时，其宽度作为统计资源，使随机初始化网络的梯度集中，而其深度作为拟合资源，更易于分离类流形，基于神经切向核及其在训练超参数化神经网络方面的作用，我们为深度全连接网络的神经切向核提供了完全优化的集中速率。