A deep equilibrium model uses implicit layers, which are implicitly defined
through an equilibrium point of an infinite sequence of computation. It avoids
any explicit computation of the infinite sequence by finding an equilibrium
point directly via root-finding and by computing gradients via implicit
differentiation. In this paper, we analyze the gradient dynamics of deep
equilibrium models with nonlinearity only on weight matrices and non-convex
objective functions of weights for regression and classification. Despite
non-convexity, convergence to global optimum at a linear rate is guaranteed
without any assumption on the width of the models, allowing the width to be
smaller than the output dimension and the number of data points. Moreover, we
prove a relation between the gradient dynamics of the deep implicit layer and
the dynamics of trust region Newton method of a shallow explicit layer. This
mathematically proven relation along with our numerical observation suggests
the importance of understanding implicit bias of implicit layers and an open
problem on the topic. Our proofs deal with implicit layers, weight tying and
nonlinearity on weights, and differ from those in the related literature.

本文基于深度平衡模型，分析其具有非凸目标函数和非线性权重矩阵的回归与分类问题的梯度动态，证明了在没有对模型宽度的任何假设的情况下会以线性速率收敛到全局最优解，同时关注了隐式层的隐式偏差和其与浅层显式层的动态的关系。