Despite Deep Learning's (DL) empirical success, our theoretical understanding
of its efficacy remains limited. One notable paradox is that while conventional
wisdom discourages perfect data fitting, deep neural networks are designed to
do just that, yet they generalize effectively. This study focuses on exploring
this phenomenon attributed to the implicit bias at play. Various sources of
implicit bias have been identified, such as step size, weight initialization,
optimization algorithm, and number of parameters. In this work, we focus on
investigating the implicit bias originating from weight initialization. To this
end, we examine the problem of solving underdetermined linear systems in
various contexts, scrutinizing the impact of initialization on the implicit
regularization when using deep networks to solve such systems. Our findings
elucidate the role of initialization in the optimization and generalization
paradoxes, contributing to a more comprehensive understanding of DL's
performance characteristics.

该研究聚焦于探索深度学习中的隐含偏差现象，具体研究了权重初始化对优化和泛化问题的影响，通过调查使用深度网络解决欠定线性系统问题时初始化的隐式正则化的作用，有助于更全面地理解深度学习的性能特点。