BriefGPT.xyz
Apr, 2023
解析训练二层ReLU网络
Training a Two Layer ReLU Network Analytically
HTML
PDF
Adrian Barbu
TL;DR
本文研究两层神经网络的ReLU激活函数和平方损失函数的优化方法，利用一种交替迭代算法寻找损失函数的关键点，实验结果显示该算法能够比随机梯度下降和Adam优化器更快、更准确地求解深度值，并且该方法没有调参困扰。
Abstract
neural networks
are usually trained with different variants of
gradient descent
based
optimization algorithms
such as stochastic
→