In this study, we investigate using graph neural network (GNN)
representations to enhance contextualized representations of pre-trained
language models (PLMs) for keyphrase extraction from lengthy documents. We show
that augmenting a PLM with graph embeddings provides a more comprehensive
semantic understanding of words in a document, particularly for long documents.
We construct a co-occurrence graph of the text and embed it using a graph
convolutional network (GCN) trained on the task of edge prediction. We propose
a graph-enhanced sequence tagging architecture that augments contextualized PLM
embeddings with graph representations. Evaluating on benchmark datasets, we
demonstrate that enhancing PLMs with graph embeddings outperforms
state-of-the-art models on long documents, showing significant improvements in
F1 scores across all the datasets. Our study highlights the potential of GNN
representations as a complementary approach to improve PLM performance for
keyphrase extraction from long documents.

本研究探讨使用图神经网络 (GNN) 表示来增强预训练语言模型 (PLMs) 的上下文表示，用于从长文档中提取关键词。我们构建了文本的共现图，并使用在边预测任务上训练的图卷积网络 (GCN) 进行嵌入。我们提出了一种增强序列标记体系结构来增强 PLM 嵌入与图嵌入。在基准数据集上评估，我们证明了使用图嵌入增强 PLM 在长文档上的性能优于现有最先进模型，并在所有数据集上显示了 F1 分数的显著改进。我们的研究突显了 GNN 表示的潜力，作为改善从长文档中提取关键词时 PLM 性能的补充方法。