In this work, we propose a new language modeling paradigm that has the
ability to perform both prediction and moderation of information flow at
multiple granularities: neural lattice language models. These models construct
a lattice of possible paths through a sentence and marginalize across this
lattice to calculate sequence probabilities or optimize parameters. This
approach allows us to seamlessly incorporate linguistic intuitions - including
polysemy and existence of multi-word lexical items - into our language model.
Experiments on multiple language modeling tasks show that English neural
lattice language models that utilize polysemous embeddings are able to improve
perplexity by 9.95% relative to a word-level baseline, and that a Chinese model
that handles multi-character tokens is able to improve perplexity by 20.94%
relative to a character-level baseline.

提出了一种名为神经格栅语言模型的新的语言建模方法，该方法在多个层次上具有信息预测和调节的能力，并通过对可能路径的格栅进行边际化以计算序列概率或优化参数。实验证明，使用多义词嵌入的英语神经格栅语言模型能够将困惑度相对于单词层面基线提高 9.95％，而处理多字符标记的中文模型能够将困惑度相对于字符层面基线提高 20.94％。