We investigate the internal representations that a recurrent neural network
(RNN) uses while learning to recognize a regular formal language. Specifically,
we train a RNN on positive and negative examples from a regular language, and
ask if there is a simple decoding function that maps states of this RNN to
states of the minimal deterministic finite automaton (MDFA) for the language.
Our experiments show that such a decoding function indeed exists, and that it
maps states of the RNN not to MDFA states, but to states of an {\em
abstraction} obtained by clustering small sets of MDFA states into
"superstates". A qualitative analysis reveals that the abstraction often has a
simple interpretation. Overall, the results suggest a strong structural
relationship between internal representations used by RNNs and finite automata,
and explain the well-known ability of RNNs to recognize formal grammatical
structure.

通过训练循环神经网络（RNN）来学习识别正则形式语言时使用的内部表示，我们研究了一个简单的解码函数，其将该 RNN 的状态映射到该语言的最小确定性有限自动机（MDFA）的状态，进而探讨了 RNN 内部表示与有限状态自动机之间的强结构关系，解释了 RNN 识别正式语法结构的能力。