Large neural language models are steadily contributing state-of-the-art
performance to question answering and other natural language and information
processing tasks. These models are expensive to train. We propose to evaluate
whether such pre-trained models can benefit from the addition of explicit
linguistics information without requiring retraining from scratch.
We present a linguistics-informed question answering approach that extends
and fine-tunes a pre-trained transformer-based neural language model with
symbolic knowledge encoded with a heterogeneous graph transformer. We
illustrate the approach by the addition of syntactic information in the form of
dependency and constituency graphic structures connecting tokens and virtual
vertices.
A comparative empirical performance evaluation with BERT as its baseline and
with Stanford Question Answering Dataset demonstrates the competitiveness of
the proposed approach. We argue, in conclusion and in the light of further
results of preliminary experiments, that the approach is extensible to further
linguistics information including semantics and pragmatics.

本研究探讨了如何在不需要从头开始的情况下，在预训练的神经语言模型中添加显式的语言学信息，以提高其性能，并提出了一种基于转换器的神经语言模型，通过异构图转换器对模型进行扩展和微调，这可以通过添加句法信息来实现，而基于斯坦福问答数据集的实证性能评估表明了该方法的竞争性。此方法包括语义和语用方面的信息，具有可扩展性。