Unsupervised pre-training on millions of digital-born or scanned documents
has shown promising advances in visual document understanding~(VDU). While
various vision-language pre-training objectives are studied in existing
solutions, the document textline, as an intrinsic granularity in VDU, has
seldom been explored so far. A document textline usually contains words that
are spatially and semantically correlated, which can be easily obtained from
OCR engines. In this paper, we propose Wukong-Reader, trained with new
pre-training objectives to leverage the structural knowledge nested in document
textlines. We introduce textline-region contrastive learning to achieve
fine-grained alignment between the visual regions and texts of document
textlines. Furthermore, masked region modeling and textline-grid matching are
also designed to enhance the visual and layout representations of textlines.
Experiments show that our Wukong-Reader has superior performance on various VDU
tasks such as information extraction. The fine-grained alignment over textlines
also empowers Wukong-Reader with promising localization ability.

本文提出的 Wukong-Reader 通过各种新的预训练目标进行训练，以利用文档文本线中嵌套的结构知识。本文还介绍了文本线 - 区域对比学习、遮罩区域建模和文本线网格匹配等方法，以增强文本线的视觉和布局表示。实验结果表明，我们的 Wukong-Reader 在各种视觉文档理解任务中具有卓越的性能，具有有前途的定位能力。