We examine the assumption that the hidden-state vectors of recurrent neural
networks (RNNs) tend to form clusters of semantically similar vectors, which we
dub the clustering hypothesis. While this hypothesis has been assumed in the
analysis of RNNs in recent years, its validity has not been studied thoroughly
on modern neural network architectures. We examine the clustering hypothesis in
the context of RNNs that were trained to recognize regular languages. This
enables us to draw on perfect ground-truth automata in our evaluation, against
which we can compare the RNN's accuracy and the distribution of the
hidden-state vectors.
We start with examining the (piecewise linear) separability of an RNN's
hidden-state vectors into semantically different classes. We continue the
analysis by computing clusters over the hidden-state vector space with multiple
state-of-the-art unsupervised clustering approaches. We formally analyze the
accuracy of computed clustering functions and the validity of the clustering
hypothesis by determining whether clusters group semantically similar vectors
to the same state in the ground-truth model.
Our evaluation supports the validity of the clustering hypothesis in the
majority of examined cases. We observed that the hidden-state vectors of
well-trained RNNs are separable, and that the unsupervised clustering
techniques succeed in finding clusters of similar state vectors.

研究表明，循环神经网络的隐藏状态向量往往会形成语义相似向量的聚类，即聚类假设；本文利用现代神经网络架构中训练出的循环神经网络，在识别正则语言的过程中，研究了聚类假设。实验结果支持聚类假设的有效性。