We establish that first-order methods avoid saddle points for almost all
initializations. Our results apply to a wide variety of first-order methods,
including gradient descent, block coordinate descent, mirror descent and
variants thereof. The connecting thread is that such algorithms can be studied
from a dynamical systems perspective in which appropriate instantiations of the
Stable Manifold Theorem allow for a global stability analysis. Thus, neither
access to second-order derivative information nor randomness beyond
initialization is necessary to provably avoid saddle points.

本研究表明，几乎所有初始化的一阶方法都可以避免鞍点问题，并且这种算法可以通过动态系统视角来研究，通过合适实例化的 Stable Manifold Theorem 进行全局稳定性分析。因此，除了初始化外，无需访问二阶导数信息或随机性即可证明避免鞍点。

一阶方法几乎总是避免鞍点

First-order Methods Almost Always Avoid Saddle Points

We show that gradient descent converges to a local minimizer, almost surely
with random initialization. This is proved by applying the Stable Manifold
Theorem from dynamical systems theory.

本文证明了随机初始化的梯度下降会收敛到局部最小值，证明使用了动力系统理论的 Stable Manifold Theorem。

梯度下降收敛于最小值点

Gradient Descent Converges to Minimizers

We formulate and prove a {\it Local Stable Manifold Theorem\/} for stochastic
differential equations (sde's) that are driven by spatial Kunita-type
semimartingales with stationary ergodic increments. Both Stratonovich and
It\^o-type equations are treated. Starting with the existence of a stochastic
flow for a sde, we introduce the notion of a hyperbolic stationary trajectory.
We prove the existence of invariant random stable and unstable manifolds in the
neighborhood of the hyperbolic stationary solution. For Stratonovich sde's, the
stable and unstable manifolds are dynamically characterized using forward and
backward solutions of the anticipating sde. The proof of the stable manifold
theorem is based on Ruelle-Oseledec multiplicative ergodic theory.

本文针对由具有平稳人人随机增量的空间 Kunita 型半鞅所驱动的随机微分方程，证明了一个局部稳定流形定理。使用 Ruelle-Oseledec 乘积符合动力系统的理论证明了关于该定理中的稳定和不稳定流形的存在证明。