Business Document Information Extraction (BDIE) is the problem of
transforming a blob of unstructured information (raw text, scanned documents,
etc.) into a structured format that downstream systems can parse and use. It
has two main tasks: Key-Information Extraction (KIE) and Line Items Recognition
(LIR). In this paper, we argue that BDIE is best modeled as a Tool Use problem,
where the tools are these downstream systems. We then present Retrieval
Augmented Structured Generation (RASG), a novel general framework for BDIE that
achieves state of the art (SOTA) results on both KIE and LIR tasks on BDIE
benchmarks.
The contributions of this paper are threefold: (1) We show, with ablation
benchmarks, that Large Language Models (LLMs) with RASG are already competitive
with or surpasses current SOTA Large Multimodal Models (LMMs) without RASG on
BDIE benchmarks. (2) We propose a new metric class for Line Items Recognition,
General Line Items Recognition Metric (GLIRM), that is more aligned with
practical BDIE use cases compared to existing metrics, such as ANLS*, DocILE,
and GriTS. (3) We provide a heuristic algorithm for backcalculating bounding
boxes of predicted line items and tables without the need for vision encoders.
Finally, we claim that, while LMMs might sometimes offer marginal performance
benefits, LLMs + RASG is oftentimes superior given real-world applications and
constraints of BDIE.

商业文档信息提取（BDIE）是将一大段非结构化信息（原始文本、扫描文档等）转换为下游系统可以解析和使用的结构化格式的问题。本文提出了一种名为 RASG 的检索增强结构化生成的新的 BDIE 框架，它在 BDIE 基准测试中实现了最先进的 KIE 任务和 LIR 任务的结果。