A neural network with one hidden layer or a two-layer network (regardless of the input layer) is the simplest feedforward neural network, whose mechanism may be the basis of more general network architectures. However, even to this type of simple architecture, it is also a ``black box''; that is, it remains unclear how to interpret the mechanism of its solutions obtained by the back-propagation algorithm and how to control the training process through a deterministic way. This paper systematically studies the first problem by constructing universal function-approximation solutions. It is shown that, both theoretically and experimentally, the training solution for the one-dimensional input could be completely understood, and that for a higher-dimensional input can also be well interpreted to some extent. Those results pave the way for thoroughly revealing the black box of two-layer ReLU networks and advance the understanding of deep ReLU networks.

本研究针对单隐层神经网络的“黑箱”问题进行系统研究，提出了通用的函数逼近解决方案。理论与实验表明，针对一维输入的训练解决方案可以完全理解，而对于高维输入也能在一定程度上进行良好解释。这项工作为揭示两层ReLU网络的黑箱机制提供了重要基础，并推进了对深层ReLU网络的理解。

单隐层ReLU网络的原理