Neural networks have a remarkable capacity for contextual processing--using
recent or nearby inputs to modify processing of current input. For example, in
natural language, contextual processing is necessary to correctly interpret
negation (e.g. phrases such as "not bad"). However, our ability to understand
how networks process context is limited. Here, we propose general methods for
reverse engineering recurrent neural networks (RNNs) to identify and elucidate
contextual processing. We apply these methods to understand RNNs trained on
sentiment classification. This analysis reveals inputs that induce contextual
effects, quantifies the strength and timescale of these effects, and identifies
sets of these inputs with similar properties. Additionally, we analyze
contextual effects related to differential processing of the beginning and end
of documents. Using the insights learned from the RNNs we improve baseline
Bag-of-Words models with simple extensions that incorporate contextual
modification, recovering greater than 90% of the RNN's performance increase
over the baseline. This work yields a new understanding of how RNNs process
contextual information, and provides tools that should provide similar insight
more broadly.

本文研究神经网络的上下文处理、不同类型的循环神经网络所产生的上下文效应，以及如何应用这些效应提高基线模型的性能。