Sequence memory is an essential attribute of natural and artificial
intelligence that enables agents to encode, store, and retrieve complex
sequences of stimuli and actions. Computational models of sequence memory have
been proposed where recurrent Hopfield-like neural networks are trained with
temporally asymmetric Hebbian rules. However, these networks suffer from
limited sequence capacity (maximal length of the stored sequence) due to
interference between the memories. Inspired by recent work on Dense Associative
Memories, we expand the sequence capacity of these models by introducing a
nonlinear interaction term, enhancing separation between the patterns. We
derive novel scaling laws for sequence capacity with respect to network size,
significantly outperforming existing scaling laws for models based on
traditional Hopfield networks, and verify these theoretical results with
numerical simulation. Moreover, we introduce a generalized pseudoinverse rule
to recall sequences of highly correlated patterns. Finally, we extend this
model to store sequences with variable timing between states' transitions and
describe a biologically-plausible implementation, with connections to motor
neuroscience.

通过引入非线性相互作用项扩增了模型的序列容量，提出了一种基于连续关联记忆的计算模型，验证了序列容量随网络规模的新的比例规律，并通过数值模拟验证了这些理论结果。此外，还引入了广义伪逆序列回忆规则，并将模型扩展到可以存储具有状态转换间隔变化的序列，提出了一种生物可信实现，与运动神经科学相关。

长序列霍普菲尔德记忆

Long Sequence Hopfield Memory

Dense Associative Memories or modern Hopfield networks permit storage and
reliable retrieval of an exponentially large (in the dimension of feature
space) number of memories. At the same time, their naive implementation is
non-biological, since it seemingly requires the existence of many-body synaptic
junctions between the neurons. We show that these models are effective
descriptions of a more microscopic (written in terms of biological degrees of
freedom) theory that has additional (hidden) neurons and only requires two-body
interactions between them. For this reason our proposed microscopic theory is a
valid model of large associative memory with a degree of biological
plausibility. The dynamics of our network and its reduced dimensional
equivalent both minimize energy (Lyapunov) functions. When certain dynamical
variables (hidden neurons) are integrated out from our microscopic theory, one
can recover many of the models that were previously discussed in the
literature, e.g. the model presented in "Hopfield Networks is All You Need"
paper. We also provide an alternative derivation of the energy function and the
update rule proposed in the aforementioned paper and clarify the relationships
between various models of this class.

提出了一种微观理论，其中只需两体相互作用，就可以有效地描述大型关联内存，具有一定的生物可信度，并实现了能量最小化的动力学。