Recently, visually-situated text parsing (VsTP) has experienced notable
advancements, driven by the increasing demand for automated document
understanding and the emergence of Generative Large Language Models (LLMs)
capable of processing document-based questions. Various methods have been
proposed to address the challenging problem of VsTP. However, due to the
diversified targets and heterogeneous schemas, previous works usually design
task-specific architectures and objectives for individual tasks, which
inadvertently leads to modal isolation and complex workflow. In this paper, we
propose a unified paradigm for parsing visually-situated text across diverse
scenarios. Specifically, we devise a universal model, called OmniParser, which
can simultaneously handle three typical visually-situated text parsing tasks:
text spotting, key information extraction, and table recognition. In
OmniParser, all tasks share the unified encoder-decoder architecture, the
unified objective: point-conditioned text generation, and the unified input &
output representation: prompt & structured sequences. Extensive experiments
demonstrate that the proposed OmniParser achieves state-of-the-art (SOTA) or
highly competitive performances on 7 datasets for the three visually-situated
text parsing tasks, despite its unified, concise design. The code is available
at this https URL

通过构建一个统一的模型 OmniParser，本文在处理不同场景的可视化文本解析问题上取得了最先进的性能，并且在文本定位、关键信息提取和表格识别任务上表现出色。