Recurrent neural network grammars (RNNG) are generative models of language
which jointly model syntax and surface structure by incrementally generating a
syntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs
achieve strong language modeling and parsing performance, but require an
annotated corpus of parse trees. In this work, we experiment with unsupervised
learning of RNNGs. Since directly marginalizing over the space of latent trees
is intractable, we instead apply amortized variational inference. To maximize
the evidence lower bound, we develop an inference network parameterized as a
neural CRF constituency parser. On language modeling, unsupervised RNNGs
perform as well their supervised counterparts on benchmarks in English and
Chinese. On constituency grammar induction, they are competitive with recent
neural language models that induce tree structures from words through attention
mechanisms.

本文介绍利用无监督学习方法，采用变分推断的深度学习算法，成功地应用于语法树和句子的生成，在语言建模和语法识别领域已经有较广泛的应用。