This paper examines gradient flow dynamics of two-homogeneous neural networks
for small initializations, where all weights are initialized near the origin.
For both square and logistic losses, it is shown that for sufficiently small
initializations, the gradient flow dynamics spend sufficient time in the
neighborhood of the origin to allow the weights of the neural network to
approximately converge in direction to the Karush-Kuhn-Tucker (KKT) points of a
neural correlation function that quantifies the correlation between the output
of the neural network and corresponding labels in the training data set. For
square loss, it has been observed that neural networks undergo saddle-to-saddle
dynamics when initialized close to the origin. Motivated by this, this paper
also shows a similar directional convergence among weights of small magnitude
in the neighborhood of certain saddle points.

本文研究了初始接近原点的两均勻神經網絡的梯度流動力學，對於方塊和邏輯損失，會有足夠長的時間在原點的附近，使神經網絡的權重大約收斂於量化神經網絡輸出和相應標籤在訓練數據集上相關性的 Karush-Kuhn-Tucker（KKT）點；方塊損失下神經網絡經常在接近原點的地方產生鞍點，本文在此基礎上展示了小幅度權重在某些鞍點附近的類似方向收斂。