We analyze the convergence of (stochastic) gradient descent algorithm for learning a convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does not rely on any specific form of the input distribution and our proofs only use the definition of ReLU, in contrast with previous works that are restricted to standard Gaussian input. We show that (stochastic) gradient descent with random initialization can learn the convolutional filter in polynomial time and the convergence rate depends on the smoothness of the input distribution and the closeness of patches. To the best of our knowledge, this is the first recovery guarantee of gradient-based algorithms for convolutional filter on non-Gaussian input distributions. Our theory also justifies the two-stage learning rate strategy in deep neural networks. While our focus is theoretical, we also present experiments that illustrate our theoretical findings.

通过随机梯度下降算法和ReLU激活函数，我们分析了卷积滤波器的收敛性。我们的研究不依赖于任何特定形式的输入分布，证明了基于梯度的算法在多项式时间内可以学习卷积滤波器，且收敛速度取决于输入分布的平滑度和卷积核的密度。此理论证明了深度神经网络中的两阶段学习率策略并提出了卷积滤波器的新的较弱恢复保证。

卷积滤波器何时易于学习？