Max-Pooling operations are a core component of deep learning architectures.
In particular, they are part of most convolutional architectures used in
machine vision, since pooling is a natural approach to pattern detection
problems. However, these architectures are not well understood from a
theoretical perspective. For example, we do not understand when they can be
globally optimized, and what is the effect of over-parameterization on
generalization. Here we perform a theoretical analysis of a convolutional
max-pooling architecture, proving that it can be globally optimized, and can
generalize well even for highly over-parameterized models. Our analysis focuses
on a data generating distribution inspired by pattern detection problem, where
a "discriminative" pattern needs to be detected among "spurious" patterns. We
empirically validate that CNNs significantly outperform fully connected
networks in our setting, as predicted by our theoretical results.

在深度学习中，池化操作是卷积网络的核心组成部分，可以自然地处理模式检测问题，但是我们没有从理论上理解何时可以进行全局优化，过参数化对泛化的影响是什么。本论文在一个受 “判别式” 和 “虚假” 的模式检测问题启发的数据生成分布下，对卷积最大池化架构进行理论分析，证明可以进行全局优化，即使是高度过参数化的模型，也可以很好地泛化，并通过实验证实，CNN 在此设置中明显优于全连接网络。