Abstract meaning representation (AMR) highlights the core semantic
information of text in a graph structure. Recently, pre-trained language models
(PLMs) have advanced tasks of AMR parsing and AMR-to-text generation,
respectively. However, PLMs are typically pre-trained on textual data, thus are
sub-optimal for modeling structural knowledge. To this end, we investigate
graph self-supervised training to improve the structure awareness of PLMs over
AMR graphs. In particular, we introduce two graph auto-encoding strategies for
graph-to-graph pre-training and four tasks to integrate text and graph
information during pre-training. We further design a unified framework to
bridge the gap between pre-training and fine-tuning tasks. Experiments on both
AMR parsing and AMR-to-text generation show the superiority of our model. To
our knowledge, we are the first to consider pre-training on semantic graphs.

本文研究了利用图形自监督训练来提高 PLMs 模型在 AMR 图结构上的结构意识； 它同时介绍了两种图形自编码策略，四个任务以及一个统一的框架来缩小预训练和微调任务之间的差距。实验结果表明了这种方法在 AMR 分析和 AMR-to-text 生成方面的优越性。