BriefGPT.xyz
Jun, 2024
神经网络在信息论极限附近通过梯度下降学习低维多项式
Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit
HTML
PDF
Jason D. Lee, Kazusato Oko, Taiji Suzuki, Denny Wu
TL;DR
通过SGD优化的两层神经网络可学习任意多项式链接函数的单指数目标函数,并具有与信息理论界限相匹配的样本和运行时间复杂度。
Abstract
We study the problem of
gradient descent learning
of a
single-index target function
$f_*(\boldsymbol{x}) = \textstyle\sigma_*\left(\langle\boldsymbol{x},\boldsymbol{\theta}\rangle\right)$ under
→