A new variant of Newton's method - named Backtracking New Q-Newton's method
(BNQN) - which has strong theoretical guarantee, is easy to implement, and has
good experimental performance, was recently introduced by the third author.
Experiments performed previously showed some remarkable properties of the
basins of attractions for finding roots of polynomials and meromorphic
functions, with BNQN. In general, they look more smooth than that of Newton's
method.
In this paper, we continue to experimentally explore in depth this remarkable
phenomenon, and connect BNQN to Newton's flow and Voronoi's diagram. This link
poses a couple of challenging puzzles to be explained. Experiments also
indicate that BNQN is more robust against random perturbations than Newton's
method and Random Relaxed Newton's method.

探索并连接了 Backtracking New Q-Newton's method（BNQN）与 Newton 的流动和 Voronoi 图的关系，并解释了其对多项式和亚纹函数的根的吸引盆地的显著现象。

回溯新 Q-Newton 方法，牛顿流，法里奥尼图与随机根查找

Backtracking New Q-Newton's method, Newton's flow, Voronoi's diagram and  Stochastic root finding

This study showcases the effectiveness of convolutional neural networks
(CNNs) in characterizing the complexity and unpredictability of basins of
attraction for diverse dynamical systems. This novel method is optimal for
exploring different parameters of dynamical systems since the conventional
methods are computationally expensive for characterizing multiple basins of
attraction. Additionally, our research includes a comparison of different CNN
architectures for this task showing the superiority of our proposed
characterization method over the conventional methods, even with obsolete
architectures.

卷积神经网络在表征多样的动力系统吸引盆复杂性和不可预测性方面的有效性进行了展示。与传统方法相比，这种新颖的方法能够以最佳方式探索动力系统的不同参数，而传统方法在表征多个吸引盆时具有计算成本较高的问题。此外，我们的研究还比较了不同卷积神经网络架构在此任务中的性能，展示了我们提出的表征方法即使使用过时的架构也优于传统方法。

基于深度学习的吸引盆分析

Deep Learning-based Analysis of Basins of Attraction

We study the properties of common loss surfaces through their Hessian matrix.
In particular, in the context of deep learning, we empirically show that the
spectrum of the Hessian is composed of two parts: (1) the bulk centered near
zero, (2) and outliers away from the bulk. We present numerical evidence and
mathematical justifications to the following conjectures laid out by Sagun et
al. (2016): Fixing data, increasing the number of parameters merely scales the
bulk of the spectrum; fixing the dimension and changing the data (for instance
adding more clusters or making the data less separable) only affects the
outliers. We believe that our observations have striking implications for
non-convex optimization in high dimensions. First, the flatness of such
landscapes (which can be measured by the singularity of the Hessian) implies
that classical notions of basins of attraction may be quite misleading. And
that the discussion of wide/narrow basins may be in need of a new perspective
around over-parametrization and redundancy that are able to create large
connected components at the bottom of the landscape. Second, the dependence of
small number of large eigenvalues to the data distribution can be linked to the
spectrum of the covariance matrix of gradients of model outputs. With this in
mind, we may reevaluate the connections within the data-architecture-algorithm
framework of a model, hoping that it would shed light into the geometry of
high-dimensional and non-convex spaces in modern applications. In particular,
we present a case that links the two observations: small and large batch
gradient descent appear to converge to different basins of attraction but we
show that they are in fact connected through their flat region and so belong to
the same basin.

我们研究了常见损失曲面的性质，并针对深度学习，通过 Hessian 矩阵的谱将其分为两个部分，并证明了 Sagun 等人所述的猜想。我们的观察结果对高维度的非凸优化具有重要意义，并提出了新的基于超参数冗余的几何角度视角。