Recurrent neural networks (RNNs) have reached striking performance in many
natural language processing tasks. This has renewed interest in whether these
generic sequence processing devices are inducing genuine linguistic knowledge.
Nearly all current analytical studies, however, initialize the RNNs with a
vocabulary of known words, and feed them tokenized input during training. We
present a multi-lingual study of the linguistic knowledge encoded in RNNs
trained as character-level language models, on input data with word boundaries
removed. These networks face a tougher and more cognitively realistic task,
having to discover any useful linguistic unit from scratch based on input
statistics. The results show that our "near tabula rasa" RNNs are mostly able
to solve morphological, syntactic and semantic tasks that intuitively
presuppose word-level knowledge, and indeed they learned, to some extent, to
track word boundaries. Our study opens the door to speculations about the
necessity of an explicit, rigid word lexicon in language learning and usage.

我们进行了一项多语言研究，探讨了在去除单词边界的输入数据上，以字符级语言模型为训练对象的循环神经网络通过基于输入统计量发现有用的语言单位这一更加困难和认知真实的任务的语言学知识编码。结果表明，我们的 “近乎于白板” 的循环神经网络主要能够解决形态、语法和语义任务，这些任务根据直观上的想象需要单词级别的知识，并且它们在某种程度上学会了追踪单词边界。我们的研究为关于语言学习和使用中明确、严格的单词词典必要性的猜测打开了大门。