Sequential activation of neurons is a common feature of network activity
during a variety of behaviors, including working memory and decision making.
Previous network models for sequences and memory emphasized specialized
architectures in which a principled mechanism is pre-wired into their
connectivity. Here we demonstrate that, starting from random connectivity and
modifying a small fraction of connections, a largely disordered recur- rent
network can produce sequences and implement working memory efficiently. We use
this process, called Partial In-Network Training (PINning), to model and match
cellular resolution imaging data from the posterior parietal cortex during a
virtual memory- guided two-alternative forced-choice task. Analysis of the
connectivity reveals that sequences propagate by the cooperation between
recurrent synaptic interactions and external inputs, rather than through
feedforward or asymmetric connections. Together our results suggest that neural
sequences may emerge through learning from largely unstructured network
architectures.

本文提出 Partial In-Network Training（PINning）方法，基于随机网络架构和少量的连接修改，实现突触交互和外部输入的协作而不是前馈或非对称连接，高效地产生神经序列和工作记忆，此方法能够匹配细胞分辨率成像数据，并暗示神经序列可能从较大结构的神经网络中学习而来。

序列生成和记忆的递归神经网络模型

Recurrent Network Models Of Sequence Generation And Memory

Neural Turing Machines (NTM) contain memory component that simulates "working
memory" in the brain to store and retrieve information to ease simple
algorithms learning. So far, only linearly organized memory is proposed, and
during experiments, we observed that the model does not always converge, and
overfits easily when handling certain tasks. We think memory component is key
to some faulty behaviors of NTM, and better organization of memory component
could help fight those problems. In this paper, we propose several different
structures of memory for NTM, and we proved in experiments that two of our
proposed structured-memory NTMs could lead to better convergence, in term of
speed and prediction accuracy on copy task and associative recall task as in
(Graves et al. 2014).

本研究通过提出多种不同 memory 的 Neural Turing Machines (NTM) 来改进传统线性存储的 NTM 的缺陷，实验证明新的结构有助于提高模型收敛速度和预测精度。