Advances in Visually Rich Document Understanding (VrDU) have enabled
information extraction and question answering over documents with complex
layouts. Two tropes of architectures have emerged -- transformer-based models
inspired by LLMs, and Graph Neural Networks. In this paper, we introduce
DocGraphLM, a novel framework that combines pre-trained language models with
graph semantics. To achieve this, we propose 1) a joint encoder architecture to
represent documents, and 2) a novel link prediction approach to reconstruct
document graphs. DocGraphLM predicts both directions and distances between
nodes using a convergent joint loss function that prioritizes neighborhood
restoration and downweighs distant node detection. Our experiments on three
SotA datasets show consistent improvement on IE and QA tasks with the adoption
of graph features. Moreover, we report that adopting the graph features
accelerates convergence in the learning process during training, despite being
solely constructed through link prediction.

该研究论文介绍了一种名为 DocGraphLM 的新型框架，它结合了预训练语言模型和图形语义，通过使用联合编码器架构来表示文档，并采用一种新颖的链接预测方法来重构文档图。DocGraphLM 通过采用图形特征，在信息提取和问答任务中持续改善了目标性能，并且提供了在训练过程中加速收敛的优点。