We present novel methods for analyzing the activation patterns of RNNs from a
linguistic point of view and explore the types of linguistic structure they
learn. As a case study, we use a multi-task gated recurrent network
architecture consisting of two parallel pathways with shared word embeddings
trained on predicting the representations of the visual scene corresponding to
an input sentence, and predicting the next word in the same sentence. Based on
our proposed method to estimate the amount of contribution of individual tokens
in the input to the final prediction of the networks we show that the image
prediction pathway: a) is sensitive to the information structure of the
sentence b) pays selective attention to lexical categories and grammatical
functions that carry semantic information c) learns to treat the same input
token differently depending on its grammatical functions in the sentence. In
contrast the language model is comparatively more sensitive to words with a
syntactic function. Furthermore, we propose methods to ex- plore the function
of individual hidden units in RNNs and show that the two pathways of the
architecture in our case study contain specialized units tuned to patterns
informative for the task, some of which can carry activations to later time
steps to encode long-term dependencies.

本文基于多任务门控循环神经网络的案例，提出了一种识别对网络最终预测贡献的关键词汇并分析 RNNs 激活模式的方法。发现该网络的语言模型层对句法功能词汇更敏感，而预测图像的层对句子的信息结构和语义信息更加敏感，并学会了根据词汇的语法功能进行不同处理，而被分化的各个隐藏单元则用于在长时间步骤中传递信息以编码长期任务相关性。