Neural dynamical systems with stable attractor structures, such as point
attractors and continuous attractors, are hypothesized to underlie meaningful
temporal behavior that requires working memory. However, working memory may not
support useful learning signals necessary to adapt to changes in the temporal
structure of the environment. We show that in addition to the continuous
attractors that are widely implicated, periodic and quasi-periodic attractors
can also support learning arbitrarily long temporal relationships. Unlike the
continuous attractors that suffer from the fine-tuning problem, the less
explored quasi-periodic attractors are uniquely qualified for learning to
produce temporally structured behavior. Our theory has broad implications for
the design of artificial learning systems and makes predictions about
observable signatures of biological neural dynamics that can support temporal
dependence learning and working memory. Based on our theory, we developed a new
initialization scheme for artificial recurrent neural networks that outperforms
standard methods for tasks that require learning temporal dynamics. Moreover,
we propose a robust recurrent memory mechanism for integrating and maintaining
head direction without a ring attractor.

基于我们的理论，我们为人工循环神经网络开发了一种新的初始化方案，优于需要学习时间动态的标准方法，并且提出了一种稳健的循环记忆机制，用于集成和维持方向信息而不需要环形吸引子。

无需连续吸引子的持续学习信号和工作记忆

Persistent learning signals and working memory without continuous  attractors

The problem of function approximation by neural dynamical systems has
typically been approached in a top-down manner: Any continuous function can be
approximated to an arbitrary accuracy by a sufficiently complex model with a
given architecture. This can lead to high-complexity controls which are
impractical in applications. In this paper, we take the opposite, constructive
approach: We impose various structural restrictions on system dynamics and
consequently characterize the class of functions that can be realized by such a
system. The systems are implemented as a cascade interconnection of a neural
stochastic differential equation (Neural SDE), a deterministic dynamical
system, and a readout map. Both probabilistic and geometric (Lie-theoretic)
methods are used to characterize the classes of functions realized by such
systems.

本文采用构造性方法对系统动力学施加各种结构限制，从而表征了这样一个可以通过神经随机微分方程（神经 SDE）、确定性动力学系统和输出映射的级联互连实现的函数类。