We address the problem of form understanding: finding text entities and the relationships/links between them in form images. The proposed FUDGE model formulates this problem on a graph of text elements (the vertices) and uses a Graph Convolutional Network to predict changes to the graph. The initial vertices are detected text lines and do not necessarily correspond to the final text entities, which can span multiple lines. Also, initial edges contain many false-positive relationships. FUDGE edits the graph structure by combining text segments (graph vertices) and pruning edges in an iterative fashion to obtain the final text entities and relationships. While recent work in this area has focused on leveraging large-scale pre-trained Language Models (LM), FUDGE achieves the same level of entity linking performance on the FUNSD dataset by learning only visual features from the (small) provided training set. FUDGE can be applied on forms where text recognition is difficult (e.g. degraded or historical forms) and on forms in resource-poor languages where pre-training such LMs is challenging. FUDGE is state-of-the-art on the historical NAF dataset.

本文介绍了一种名为 FUDGE 的模型，利用图神经网络预测文本元素之间关系、从而基于迭代方式，组合文本段落并删除边缘以得到最终的文本实体和关系。与最近在该领域的大量预训练语言模型相比，FUDGE 从（小型的）提供的训练集学习视觉特征，能够适用于文本识别困难的表格以及资源匮乏的语言。在历史 NAF 数据集上的结果表明 FUDGE 是最先进的方法。

动态图编辑的形态理解