We examine the extent to which, in principle, linguistic graph
representations can complement and improve neural language modeling. With an
ensemble setup consisting of a pretrained Transformer and ground-truth graphs
from one of 7 different formalisms, we find that, overall, semantic
constituency structures are most useful to language modeling performance --
outpacing syntactic constituency structures as well as syntactic and semantic
dependency structures. Further, effects vary greatly depending on
part-of-speech class. In sum, our findings point to promising tendencies in
neuro-symbolic language modeling and invite future research quantifying the
design choices made by different formalisms.

本文探讨了语言图表示在理论上能否 complement 并提高神经语言建模的能力。通过集成一个预训练的 Transformer 和七种不同形式主义的 ground-truth 图，研究发现，总体而言，语义组成结构对于语言建模的性能最有用，超越了句法组成结构以及句法和语义依存关系结构。此外，这种效应在不同的词性类别中差异很大。总之，我们的研究结果为神经符号语言建模带来了有前途的倾向，并邀请未来研究 quantifying 不同形式主义所做的设计选择。