Existing methods for Visual Information Extraction (VIE) from form-like
documents typically fragment the process into separate subtasks, such as key
information extraction, key-value pair extraction, and choice group extraction.
However, these approaches often overlook the hierarchical structure of form
documents, including hierarchical key-value pairs and hierarchical choice
groups. To address these limitations, we present a new perspective, reframing
VIE as a relation prediction problem and unifying labels of different tasks
into a single label space. This unified approach allows for the definition of
various relation types and effectively tackles hierarchical relationships in
form-like documents. In line with this perspective, we present UniVIE, a
unified model that addresses the VIE problem comprehensively. UniVIE functions
using a coarse-to-fine strategy. It initially generates tree proposals through
a tree proposal network, which are subsequently refined into hierarchical trees
by a relation decoder module. To enhance the relation prediction capabilities
of UniVIE, we incorporate two novel tree constraints into the relation decoder:
a tree attention mask and a tree level embedding. Extensive experimental
evaluations on both our in-house dataset HierForms and a publicly available
dataset SIBR, substantiate that our method achieves state-of-the-art results,
underscoring the effectiveness and potential of our unified approach in
advancing the field of VIE.

现有的视觉信息提取方法通常将从表单样式文档中提取信息的过程分解为不同的子任务，我们提出了一种新的视角，将其重新定义为关系预测问题，并统一了不同任务的标签，以解决表单文档中的层次关系。我们提出的统一模型 UniVIE 通过构建树提案和关系解码模块来全面解决视觉信息提取问题，通过引入树注意力掩码和树级别嵌入来增强关系预测能力。在我们的数据集 HierForms 和公开数据集 SIBR 上进行的广泛实验评估证明了我们方法达到了最先进的结果，突显了我们统一方法在推进视觉信息提取领域的有效性和潜力。

UniVIE：一种基于统一标签空间的表单式文档图像信息提取方法

UniVIE: A Unified Label Space Approach to Visual Information Extraction  from Form-like Documents

Automating information extraction from form-like documents at scale is a
pressing need due to its potential impact on automating business workflows
across many industries like financial services, insurance, and healthcare. The
key challenge is that form-like documents in these business workflows can be
laid out in virtually infinitely many ways; hence, a good solution to this
problem should generalize to documents with unseen layouts and languages. A
solution to this problem requires a holistic understanding of both the textual
segments and the visual cues within a document, which is non-trivial. While the
natural language processing and computer vision communities are starting to
tackle this problem, there has not been much focus on (1) data-efficiency, and
(2) ability to generalize across different document types and languages.
In this paper, we show that when we have only a small number of labeled
documents for training (~50), a straightforward transfer learning approach from
a considerably structurally-different larger labeled corpus yields up to a 27
F1 point improvement over simply training on the small corpus in the target
domain. We improve on this with a simple multi-domain transfer learning
approach, that is currently in production use, and show that this yields up to
a further 8 F1 point improvement. We make the case that data efficiency is
critical to enable information extraction systems to scale to handle hundreds
of different document-types, and learning good representations is critical to
accomplishing this.

本文介绍了如何使用少量标记数据和被标记语料库进行转移学习，以实现在规模上自动从表单状文件提取信息。