Recent improvements in KG-to-text generation are due to additional auxiliary
pre-training tasks designed to give the fine-tune task a boost in performance.
These tasks require extensive computational resources while only suggesting
marginal improvements. Here, we demonstrate that by fusing graph-aware elements
into existing pre-trained language models, we are able to outperform
state-of-the-art models and close the gap imposed by additional pre-training
tasks. We do so by proposing a mask structure to capture neighborhood
information and a novel type encoder that adds a bias to the graph-attention
weights depending on the connection type. Experiments on two KG-to-text
benchmark datasets show our models are competitive while involving fewer
parameters and no additional pre-training tasks. By formulating the problem as
a framework, we can interchange the various proposed components and begin
interpreting KG-to-text generative models based on the topological and type
information found in a graph.

通过将图感知元素融合到现有的预训练语言模型中，提出了一种掩蔽结构来捕获邻域信息和一种新型类型编码器，以便于基于图中的拓扑和类型信息解释 KG-to-text 生成模型（在不需要额外的预训练任务的情况下），实验结果表明我们的模型具有较高的竞争力。