This work addresses weight optimization problem for fully-connected
feed-forward neural networks. Unlike existing approaches that are based on
back-propagation (BP) and chain rule gradient-based optimization (which implies
iterative execution, potentially burdensome and time-consuming in some cases),
the proposed approach offers the solution for weight optimization in
closed-form by means of least squares (LS) methodology. In the case where the
input-to-output mapping is injective, the new approach optimizes the weights in
a back-propagating fashion in a single iteration by jointly optimizing a set of
weights in each layer for each neuron. In the case where the input-to-output
mapping is not injective (e.g., in classification problems), the proposed
solution is easily adapted to obtain its final solution in a few iterations. An
important advantage over the existing solutions is that these computations (for
all neurons in a layer) are independent from each other; thus, they can be
carried out in parallel to optimize all weights in a given layer
simultaneously. Furthermore, its running time is deterministic in the sense
that one can obtain the exact number of computations necessary to optimize the
weights in all network layers (per iteration, in the case of non-injective
mapping). Our simulation and empirical results show that the proposed scheme,
BPLS, works well and is competitive with existing ones in terms of accuracy,
but significantly surpasses them in terms of running time. To summarize, the
new method is straightforward to implement, is competitive and computationally
more efficient than the existing ones, and is well-tailored for parallel
implementation.

该研究针对全连接前馈神经网络的权重优化问题，提出了使用最小二乘法来实现权重优化的方法。这种方法具有闭合形式解，能够在单次迭代中通过反向传播的方式优化每层神经元的权重，并且能够在几次迭代中适应输入到输出映射不是单射的情况。与现有解决方案相比，这种方法的计算是并行的，具有确定性的运行时间，并且在准确性方面与现有解决方案竞争，但在运行时间方面明显优于它们。总之，这种新方法实现简单，与现有方法相比在计算效率上更高，并且适于并行实现。