Prior study shows that pre-training techniques can boost the performance of
visual document understanding (VDU), which typically requires models to gain
abilities to perceive and reason both document texts and layouts (e.g.,
locations of texts and table-cells). To this end, we propose visually guided
generative text-layout pre-training, named ViTLP. Given a document image, the
model optimizes hierarchical language and layout modeling objectives to
generate the interleaved text and layout sequence. In addition, to address the
limitation of processing long documents by Transformers, we introduce a
straightforward yet effective multi-segment generative pre-training scheme,
facilitating ViTLP to process word-intensive documents of any length. ViTLP can
function as a native OCR model to localize and recognize texts of document
images. Besides, ViTLP can be effectively applied to various downstream VDU
tasks. Extensive experiments show that ViTLP achieves competitive performance
over existing baselines on benchmark VDU tasks, including information
extraction, document classification, and document question answering.

给定一张文档图像，本研究提出了一种名为 ViTLP 的视觉引导生成文本 - 布局预训练模型，通过生成交错文本和布局序列来优化层次化语言和布局建模目标，以处理任意长度的文字密集型文档，并有效应用于各种下游的视觉文档理解任务。