BriefGPT.xyz
Mar, 2022
宽神经网络的线性转换是由组装弱模型的新性质
Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models
HTML
PDF
Chaoyue Liu, Libin Zhu, Mikhail Belkin
TL;DR
该论文探讨了宽神经网络与线性输出层的关系，并发现它们在梯度下降的优化路径区域中几乎是线性的，并且有几乎恒定的神经切向核。通过递归地构建由一组子模型组成的神经网络装配模型的方法，该论文提出了一种新的视角解释宽神经网络的线性结构。
Abstract
Wide
neural networks
with linear output layer have been shown to be near-linear, and to have near-constant
neural tangent kernel
(NTK), in a region containing the
→