We prove that training neural networks on 1-D data is equivalent to solving a convex Lasso problem with a fixed, explicitly defined dictionary matrix of features. The specific dictionary depends on the activation and depth. We consider 2-layer networks with piecewise linear activations, deep narrow ReLU networks with up to 4 layers, and rectangular and tree networks with sign activation and arbitrary depth. Interestingly in ReLU networks, a fourth layer creates features that represent reflections of training data about themselves. The Lasso representation sheds insight to globally optimal networks and the solution landscape.

通过证明，我们发现在一维数据上训练神经网络等效于求解一个带有固定、明确定义的特征字典矩阵的凸Lasso问题。具体的字典取决于激活函数和深度。我们研究了具有分段线性激活函数的两层网络，具有最多4层的深窄ReLU网络，以及具有符号激活和任意深度的矩形和树状网络。有趣的是，在ReLU网络中，第四层创建了代表训练数据关于它们自身的反射的特征。Lasso表示法为全局最优网络和解决方案空间提供了洞察。

一个镜面图书馆：低维深度神经网络是具有反射特征的凸Lasso模型