The field of visual document understanding has witnessed a rapid growth in
emerging challenges and powerful multi-modal strategies. However, they rely on
an extensive amount of document data to learn their pretext objectives in a
``pre-train-then-fine-tune'' paradigm and thus, suffer a significant
performance drop in real-world online industrial settings. One major reason is
the over-reliance on OCR engines to extract local positional information within
a document page. Therefore, this hinders the model's generalizability,
flexibility and robustness due to the lack of capturing global information
within a document image. We introduce TransferDoc, a cross-modal
transformer-based architecture pre-trained in a self-supervised fashion using
three novel pretext objectives. TransferDoc learns richer semantic concepts by
unifying language and visual representations, which enables the production of
more transferable models. Besides, two novel downstream tasks have been
introduced for a ``closer-to-real'' industrial evaluation scenario where
TransferDoc outperforms other state-of-the-art approaches.

TransferDoc 是跨模态 transformer 架构，通过在自我监督学习中使用三个新颖的预训练目标，实现更丰富的语义概念的学习，从而在工业评估场景中胜过其他最先进的方法。