Deep learning has shown high performances in various types of tasks from visual recognition to natural language processing, which indicates superior flexibility and adaptivity of deep learning. To understand this phenomenon theoretically, we develop a new approximation and estimation error analysis of deep learning with the ReLU activation for functions in a Besov space and its variant with mixed smoothness. The Besov space is a considerably general function space including the Holder space and Sobolev space, and especially can capture spatial inhomogeneity of smoothness. Through the analysis in the Besov space, it is shown that deep learning can achieve the minimax optimal rate and outperform any non-adaptive (linear) estimator such as kernel ridge regression, which shows that deep learning has higher adaptivity to the spatial inhomogeneity of the target function than other estimators such as linear ones. In addition to this, it is shown that deep learning can avoid the curse of dimensionality if the target function is in a mixed smooth Besov space. We also show that the dependency of the convergence rate on the dimensionality is tight due to its minimax optimality. These results support high adaptivity of deep learning and its superior ability as a feature extractor.

通过对 ReLU 激活的深度学习在 Besov 空间及其变体中的函数进行近似和估计误差分析，可以得出深度学习可以实现最小最优速率并胜过其他线性估计器。同时，它可以避免在混合平滑 Besov 空间中出现维度诅咒，这些结果支持了深度学习在特征提取方面的卓越能力和高度适应性。

基于深度ReLU网络的Besov空间和混合光滑Besov空间学习的自适应性：最优速率和维数诅咒