While state-of-the-art neural network models continue to achieve lower
perplexity scores on language modeling benchmarks, it remains unknown whether
optimizing for broad-coverage predictive performance leads to human-like
syntactic knowledge. Furthermore, existing work has not provided a clear
picture about the model properties required to produce proper syntactic
generalizations. We present a systematic evaluation of the syntactic knowledge
of neural language models, testing 20 combinations of model types and data
sizes on a set of 34 English-language syntactic test suites. We find
substantial differences in syntactic generalization performance by model
architecture, with sequential models underperforming other architectures.
Factorially manipulating model architecture and training dataset size (1M--40M
words), we find that variability in syntactic generalization performance is
substantially greater by architecture than by dataset size for the corpora
tested in our experiments. Our results also reveal a dissociation between
perplexity and syntactic generalization performance.

我们对神经语言模型的句法知识进行了系统评估，发现模型架构的不同对句法泛化性能产生了显著影响，而不同数据集大小的影响相对较低。