通过浅层ReLU网络学习神经元：相关输入的动力学和隐式偏置

Jun, 2023

Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs

Dmitry Chistikov, Matthias Englert, Ranko Lazic

TL;DR通过训练一个从一个小初始值开始的任意宽度的一层ReLU神经网络来证明，对于学习单个神经元的基本回归任务，该网络能收敛于零损失并隐含有利于最小化网络参数秩的偏见。

Abstract

We prove that, for the fundamental regression task of learning a single neuron, training a one-hidden layer relu network of any width by <