Key information extraction (KIE) from document images requires understanding
the contextual and spatial semantics of texts in two-dimensional (2D) space.
Many recent studies try to solve the task by developing pre-trained language
models focusing on combining visual features from document images with texts
and their layout. On the other hand, this paper tackles the problem by going
back to the basic: effective combination of text and layout. Specifically, we
propose a pre-trained language model, named BROS (BERT Relying On Spatiality),
that encodes relative positions of texts in 2D space and learns from unlabeled
documents with area-masking strategy. With this optimized training scheme for
understanding texts in 2D space, BROS shows comparable or better performance
compared to previous methods on four KIE benchmarks (FUNSD, SROIE*, CORD, and
SciTSR) without relying on visual features. This paper also reveals two
real-world challenges in KIE tasks-(1) minimizing the error from incorrect text
ordering and (2) efficient learning from fewer downstream examples-and
demonstrates the superiority of BROS over previous methods. Code is available
at this https URL

提出一种名为 BROS（BERT Relying On Spatiality）的预训练语言模型，重点在于将文字和布局有效组合起来，通过对文档中文字的相对位置进行编码以学习二维空间中的文本语义和布局，并利用面积屏蔽策略从未标记文档进行优化训练。BROS 表现出与其他方法相当甚至更好的性能，解决了两个实际的关键信息提取问题：减少文本顺序错误和使用较少的下游示例进行高效学习。