Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, while neglecting cell-level layout information that is important for form image understanding. In this paper, we propose a new pre-training approach, StructuralLM, to jointly leverage cell and layout information from scanned documents. Specifically, we pre-train StructuralLM with two new designs to make the most of the interactions of cell and layout information: 1) each cell as a semantic unit; 2) classification of cell positions. The pre-trained StructuralLM achieves new state-of-the-art results in different types of downstream tasks, including form understanding (from 78.95 to 85.14), document visual question answering (from 72.59 to 83.94) and document image classification (from 94.43 to 96.08).

本文提出了一种新的预训练方法 StructuralLM，可以联合使用扫描文档中的单元格和布局信息，在对下游自然语言处理任务进行微调时获得了新的最先进结果，包括通过分类单元格位置等两种新方法进行预处理。能够有效提高表格理解、文档可视化问答和文档图像分类方面的表现。

StructuralLM：面向形式理解的结构化预训练