In this paper, we aim at developing scalable neural network-type learning
systems. Motivated by the idea of "constructive neural networks" in
approximation theory, we focus on "constructing" rather than "training"
feed-forward neural networks (FNNs) for learning, and propose a novel FNNs
learning system called the constructive feed-forward neural network (CFN).
Theoretically, we prove that the proposed method not only overcomes the
classical saturation problem for FNN approximation, but also reaches the
optimal learning rate when the regression function is smooth, while the
state-of-the-art learning rates established for traditional FNNs are only near
optimal (up to a logarithmic factor). A series of numerical simulations are
provided to show the efficiency and feasibility of CFN via comparing with the
well-known regularized least squares (RLS) with Gaussian kernel and extreme
learning machine (ELM).

本文提出了一种新的前馈神经网络学习系统 —— 构造性前馈神经网络（CFN），并证明了该方法不仅克服了传统 FNN 逼近时的饱和问题，而且在回归函数光滑时达到了最优学习率。通过与正则化最小二乘法（RLS）和极限学习机（ELM）进行比较，数值模拟表明了 CFN 的高效性和可行性。