We investigate the optimization dynamics of gradient descent in a non-convex
and high-dimensional setting, with a focus on the phase retrieval problem as a
case study for complex loss landscapes. We first study the high-dimensional
limit where both the number $M$ and the dimension $N$ of the data are going to
infinity at fixed signal-to-noise ratio $\alpha = M/N$. By analyzing how the
local curvature changes during optimization, we uncover that for intermediate
$\alpha$, the Hessian displays a downward direction pointing towards good
minima in the first regime of the descent, before being trapped in bad minima
at the end. Hence, the local landscape is benign and informative at first,
before gradient descent brings the system into a uninformative maze. The
transition between the two regimes is associated to a BBP-type threshold in the
time-dependent Hessian. Through both theoretical analysis and numerical
experiments, we show that in practical cases, i.e. for finite but even very
large $N$, successful optimization via gradient descent in phase retrieval is
achieved by falling towards the good minima before reaching the bad ones. This
mechanism explains why successful recovery is obtained well before the
algorithmic transition corresponding to the high-dimensional limit.
Technically, this is associated to strong logarithmic corrections of the
algorithmic transition at large $N$ with respect to the one expected in the
$N\to\infty$ limit. Our analysis sheds light on such a new mechanism that
facilitate gradient descent dynamics in finite large dimensions, also
highlighting the importance of good initialization of spectral properties for
optimization in complex high-dimensional landscapes.

我们研究了非凸和高维环境中梯度下降的优化动力学，重点研究了相位恢复问题作为复杂损失地形的案例研究。我们通过分析优化过程中局部曲率的变化，发现在下降的第一个阶段中，对于中等信噪比，Hessian 矩阵显示出朝向好的极小值的下降方向，然后被困在坏的极小值中。成功的相位恢复通过梯度下降在达到坏的极小值之前朝向好的极小值实现，这种机制解释了为什么在高维极限对应的算法过渡之前就能成功恢复。我们的分析揭示了这种新机制，在有限但非常大的维度下促进梯度下降动力学，同时强调了初始化谱特性对于在复杂高维地形中的优化的重要性。