In recent years, the use of multi-modal pre-trained Transformers has led to
significant advancements in visually-rich document understanding. However,
existing models have mainly focused on features such as text and vision while
neglecting the importance of layout relationship between text nodes. In this
paper, we propose GraphLayoutLM, a novel document understanding model that
leverages the modeling of layout structure graph to inject document layout
knowledge into the model. GraphLayoutLM utilizes a graph reordering algorithm
to adjust the text sequence based on the graph structure. Additionally, our
model uses a layout-aware multi-head self-attention layer to learn document
layout knowledge. The proposed model enables the understanding of the spatial
arrangement of text elements, improving document comprehension. We evaluate our
model on various benchmarks, including FUNSD, XFUND and CORD, and achieve
state-of-the-art results among these datasets. Our experimental results
demonstrate that our proposed method provides a significant improvement over
existing approaches and showcases the importance of incorporating layout
information into document understanding models. We also conduct an ablation
study to investigate the contribution of each component of our model. The
results show that both the graph reordering algorithm and the layout-aware
multi-head self-attention layer play a crucial role in achieving the best
performance.

我们提出了 GraphLayoutLM 模型，它利用布局结构图的建模将文档布局知识注入模型，使得模型能够理解文本元素的空间排列，以提高文档的理解能力，并在 FUNSD、XFUND 和 CORD 等基准测试中取得了最先进的结果。