We propose an algorithm for optimizing the parameters of single hidden layer
neural networks. Specifically, we derive a blockwise difference-of-convex (DC)
functions representation of the objective function. Based on the latter, we
propose a block coordinate descent (BCD) approach that we combine with a
tailored difference-of-convex functions algorithm (DCA). We prove global
convergence of the proposed algorithm. Furthermore, we mathematically analyze
the convergence rate of parameters and the convergence rate in value (i.e., the
training loss). We give conditions under which our algorithm converges linearly
or even faster depending on the local shape of the loss function. We confirm
our theoretical derivations numerically and compare our algorithm against
state-of-the-art gradient-based solvers in terms of both training loss and test
loss.

我们提出了一种优化单隐藏层神经网络参数的算法，其中我们导出了目标函数的分块凸差（DC）函数表示。基于后者，我们提出了一种分块坐标下降（BCD）方法，将其与定制的凸差函数算法（DCA）结合起来。我们证明了所提算法的全局收敛性。此外，我们在理论上分析了参数的收敛速度和值的收敛速度（即训练损失）。我们给出了算法收敛线性或甚至更快的条件，取决于损失函数的局部形状。我们通过数值实验验证了理论推导的正确性，并在训练损失和测试损失方面将我们的算法与最先进的基于梯度的求解器进行了比较。