We present DocFormer -- a multi-modal transformer based architecture for the
task of Visual Document Understanding (VDU). VDU is a challenging problem which
aims to understand documents in their varied formats (forms, receipts etc.) and
layouts. In addition, DocFormer is pre-trained in an unsupervised fashion using
carefully designed tasks which encourage multi-modal interaction. DocFormer
uses text, vision and spatial features and combines them using a novel
multi-modal self-attention layer. DocFormer also shares learned spatial
embeddings across modalities which makes it easy for the model to correlate
text to visual tokens and vice versa. DocFormer is evaluated on 4 different
datasets each with strong baselines. DocFormer achieves state-of-the-art
results on all of them, sometimes beating models 4x its size (in no. of
parameters).

本文介绍了基于多模态变压器的 DocFormer 架构，它以无监督预训练方式运行，能够更好地理解各种格式和布局的文档，并结合了文本、视觉和空间特征，实现了多模态自注意层和共享学习空间嵌入等特点，使其在四个数据集上均取得了业界领先的结果。