We theoretically discuss why deep neural networks (DNNs) performs better than
other models in some cases by investigating statistical properties of DNNs for
non-smooth functions. While DNNs have empirically shown higher performance than
other standard methods, understanding its mechanism is still a challenging
problem. From an aspect of the statistical theory, it is known many standard
methods attain the optimal rate of generalization errors for smooth functions
in large sample asymptotics, and thus it has not been straightforward to find
theoretical advantages of DNNs. This paper fills this gap by considering
learning of a certain class of non-smooth functions, which was not covered by
the previous theory. We derive the generalization error of estimators by DNNs
with a ReLU activation, and show that convergence rates of the generalization
by DNNs are almost optimal to estimate the non-smooth functions, while some of
the popular models do not attain the optimal rate. In addition, our theoretical
result provides guidelines for selecting an appropriate number of layers and
edges of DNNs. We provide numerical experiments to support the theoretical
results.

本文阐述了深度神经网络在一定情况下为何比其他模型表现更好，并通过考虑一定类别的非光滑函数，推导了使用 ReLU 激活的 DNN 的估计器的泛化误差，同时说明了 DNN 的收敛速率几乎是最优的，而某些流行的模型则未达到最优速率，这为选择合适的 DNN 层数和边提供了指导。