Despite the superior performance of large language models to generate natural
language texts, it is hard to generate texts with correct logic according to a
given task, due to the difficulties for neural models to capture implied rules
from free-form texts. In this paper, we propose a novel graph-based language
model, Logical-GLM, to infuse logic into language models for more valid text
generation and interpretability. Specifically, we first capture information
from natural language instructions and construct logical bayes graphs that
generally describe domains. Next, we generate logical skeletons to guide
language model training, infusing domain knowledge into language models.
Finally, we alternately optimize the searching policy of graphs and language
models until convergence. The experimental results show that Logical-GLM is
both effective and efficient compared with traditional language models, despite
using smaller-scale training data and fewer parameters. Our approach can
generate instructional texts with more correct logic owing to the internalized
domain knowledge. Moreover, the usage of logical graphs reflects the inner
mechanism of the language models, which improves the interpretability of
black-box models.

将逻辑注入语言模型的新型基于图的语言模型 Logical-GLM，通过捕捉自由形式文本中所包含的隐含规则，并生成逻辑骨架以指导语言模型的训练，从而实现更准确的文本生成和可解释性，并通过使用逻辑图反映语言模型的内在机制提高黑盒模型的可解释性。