Although deep convolutional networks have achieved improved performance in
many natural language tasks, they have been treated as black boxes because they
are difficult to interpret. Especially, little is known about how they
represent language in their intermediate layers. In an attempt to understand
the representations of deep convolutional networks trained on language tasks,
we show that individual units are selectively responsive to specific morphemes,
words, and phrases, rather than responding to arbitrary and uninterpretable
patterns. In order to quantitatively analyze such an intriguing phenomenon, we
propose a concept alignment method based on how units respond to the replicated
text. We conduct analyses with different architectures on multiple datasets for
classification and translation tasks and provide new insights into how deep
models understand natural language.

本文尝试理解深度卷积神经网络对自然语言任务训练时的表示方式，发现其各个单元对特定词素、单词和短语具有选择性响应，而非对任意难以解释的模式响应。作者提出了基于复制文本单位响应的概念对齐方法，对多个数据集的分类和翻译任务进行了定量分析，并为我们深入理解深度模型如何理解自然语言提供了新思路。