Key Information Extraction (KIE) is a challenging multimodal task that aims
to extract structured value semantic entities from visually rich documents.
Although significant progress has been made, there are still two major
challenges that need to be addressed. Firstly, the layout of existing datasets
is relatively fixed and limited in the number of semantic entity categories,
creating a significant gap between these datasets and the complex real-world
scenarios. Secondly, existing methods follow a two-stage pipeline strategy,
which may lead to the error propagation problem. Additionally, they are
difficult to apply in situations where unseen semantic entity categories
emerge. To address the first challenge, we propose a new large-scale
human-annotated dataset named Complex Layout form for key information
EXtraction (CLEX), which consists of 5,860 images with 1,162 semantic entity
categories. To solve the second challenge, we introduce Parallel Pointer-based
Network (PPN), an end-to-end model that can be applied in zero-shot and
few-shot scenarios. PPN leverages the implicit clues between semantic entities
to assist extracting, and its parallel extraction mechanism allows it to
extract multiple results simultaneously and efficiently. Experiments on the
CLEX dataset demonstrate that PPN outperforms existing state-of-the-art methods
while also offering a much faster inference speed.

提出了一个挑战性的多模态任务：关键信息提取 (Key Information Extraction)，旨在从视觉丰富的文档中提取结构化的值语义实体。通过引入一种新的大规模人工标注数据集，名为复杂布局表单 (CLEX)，该数据集包括 5,860 个图像和 1,162 个语义实体类别，以解决现有数据集布局相对固定且语义实体类别有限的问题。通过引入一种名为 Parallel Pointer-based Network (PPN) 的端到端模型，用于零样本和小样本场景，解决了现有方法在错误传播和未知语义实体类别问题上的困难。CLEX 数据集上的实验证明，PPN 在推理速度方面表现出色，并且优于现有的最先进方法。