A continuous one-dimensional map with period three includes all periods. This
raises the following question: Can we obtain any types of periodic orbits
solely by learning three data points? We consider learning period three with
random neural networks and report the universal property associated with it. We
first show that the trained networks have a thermodynamic limit that depends on
the choice of target data and network settings. Our analysis reveals that
almost all learned periods are unstable and each network has its characteristic
attractors (which can even be untrained ones). Here, we propose the concept of
characteristic bifurcation expressing embeddable attractors intrinsic to the
network, in which the target data points and the scale of the network weights
function as bifurcation parameters. In conclusion, learning period three
generates various attractors through characteristic bifurcation due to the
stability change in latently existing numerous unstable periods of the system.

学习周期三的过程通过特征分叉产生各种吸引子，这是由于系统中潜在存在的众多不稳定周期引起的稳定性变化。

学时三中的热力学极限

Thermodynamic limit in learning period three

Neural dynamical systems with stable attractor structures, such as point
attractors and continuous attractors, are hypothesized to underlie meaningful
temporal behavior that requires working memory. However, working memory may not
support useful learning signals necessary to adapt to changes in the temporal
structure of the environment. We show that in addition to the continuous
attractors that are widely implicated, periodic and quasi-periodic attractors
can also support learning arbitrarily long temporal relationships. Unlike the
continuous attractors that suffer from the fine-tuning problem, the less
explored quasi-periodic attractors are uniquely qualified for learning to
produce temporally structured behavior. Our theory has broad implications for
the design of artificial learning systems and makes predictions about
observable signatures of biological neural dynamics that can support temporal
dependence learning and working memory. Based on our theory, we developed a new
initialization scheme for artificial recurrent neural networks that outperforms
standard methods for tasks that require learning temporal dynamics. Moreover,
we propose a robust recurrent memory mechanism for integrating and maintaining
head direction without a ring attractor.

基于我们的理论，我们为人工循环神经网络开发了一种新的初始化方案，优于需要学习时间动态的标准方法，并且提出了一种稳健的循环记忆机制，用于集成和维持方向信息而不需要环形吸引子。

无需连续吸引子的持续学习信号和工作记忆

Persistent learning signals and working memory without continuous  attractors

Compared to ordinary function minimization problems, min-max optimization
algorithms encounter far greater challenges because of the existence of
periodic cycles and similar phenomena. Even though some of these behaviors can
be overcome in the convex-concave regime, the general case is considerably more
difficult. On that account, we take an in-depth look at a comprehensive class
of state-of-the art algorithms and prevalent heuristics in non-convex /
non-concave problems, and we establish the following general results: a)
generically, the algorithms' limit points are contained in the ICT sets of a
common, mean-field system; b) the attractors of this system also attract the
algorithms in question with arbitrarily high probability; and c) all algorithms
avoid the system's unstable sets with probability 1. On the surface, this
provides a highly optimistic outlook for min-max algorithms; however, we show
that there exist spurious attractors that do not contain any stationary points
of the problem under study. In this regard, our work suggests that existing
min-max algorithms may be subject to inescapable convergence failures. We
complement our theoretical analysis by illustrating such attractors in simple,
two-dimensional, almost bilinear problems.

本文研究了最小 - 最大优化算法在非凸 /non-concave 问题中的表现，发现其极限点包含在共同的平均场系统的 ICT 集合中，其吸引子与算法相关，但存在不包含问题静止点的虚假吸引子，这可能导致算法收敛失败。