Understanding the intricate operations of Recurrent Neural Networks (RNNs) mechanistically is pivotal for advancing their capabilities and applications. In this pursuit, we propose the Episodic Memory Theory (EMT), illustrating that RNNs can be conceptualized as discrete-time analogs of the recently proposed General Sequential Episodic Memory Model. To substantiate EMT, we introduce a novel set of algorithmic tasks tailored to probe the variable binding behavior in RNNs. Utilizing the EMT, we formulate a mathematically rigorous circuit that facilitates variable binding in these tasks. Our empirical investigations reveal that trained RNNs consistently converge to the variable binding circuit, thus indicating universality in the dynamics of RNNs. Building on these findings, we devise an algorithm to define a privileged basis, which reveals hidden neurons instrumental in the temporal storage and composition of variables, a mechanism vital for the successful generalization in these tasks. We show that the privileged basis enhances the interpretability of the learned parameters and hidden states of RNNs. Our work represents a step toward demystifying the internal mechanisms of RNNs and, for computational neuroscience, serves to bridge the gap between artificial neural networks and neural memory models.

我们提出了“情节记忆理论”，旨在揭示循环神经网络作为离散时间模拟的“一般顺序情节记忆模型”，我们引入了一套适用于探索循环神经网络中变量绑定行为的新颖算法任务，并且通过数学上严格的电路形式加强了这些任务中的变量绑定。我们的实验研究表明，经过训练的循环神经网络始终收敛到变量绑定电路，从而揭示了循环神经网络动力学的普遍性，进而设计了一种算法来定义特权基础，这揭示了在时间存储和组合变量中起关键作用的隐藏神经元，这一机制对于成功的泛化至关重要。我们展示了特权基础如何提高了循环神经网络的学习参数和隐藏状态的可解释性。我们的工作在循环神经网络的内部机制解谜方面迈出了一步，并且对计算神经科学有助于弥合人工神经网络和神经记忆模型之间的差距。

递归神经网络的机制性解释的记忆片段理论