We propose a novel end-to-end document understanding model called SeRum
(SElective Region Understanding Model) for extracting meaningful information
from document images, including document analysis, retrieval, and office
automation.
Unlike state-of-the-art approaches that rely on multi-stage technical schemes
and are computationally expensive,
SeRum converts document image understanding and recognition tasks into a
local decoding process of the visual tokens of interest, using a content-aware
token merge module.
This mechanism enables the model to pay more attention to regions of interest
generated by the query decoder, improving the model's effectiveness and
speeding up the decoding speed of the generative scheme.
We also designed several pre-training tasks to enhance the understanding and
local awareness of the model.
Experimental results demonstrate that SeRum achieves state-of-the-art
performance on document understanding tasks and competitive results on text
spotting tasks.
SeRum represents a substantial advancement towards enabling efficient and
effective end-to-end document understanding.

我们提出了一种新颖的端到端文档理解模型，称为 SeRum（选择性区域理解模型），用于从文档图像中提取有意义的信息，包括文档分析、检索和办公自动化。