Building document-grounded dialogue systems have received growing interest as
documents convey a wealth of human knowledge and commonly exist in enterprises.
Wherein, how to comprehend and retrieve information from documents is a
challenging research problem. Previous work ignores the visual property of
documents and treats them as plain text, resulting in incomplete modality. In
this paper, we propose a Layout-aware document-level Information Extraction
dataset, LIE, to facilitate the study of extracting both structural and
semantic knowledge from visually rich documents (VRDs), so as to generate
accurate responses in dialogue systems. LIE contains 62k annotations of three
extraction tasks from 4,061 pages in product and official documents, becoming
the largest VRD-based information extraction dataset to the best of our
knowledge. We also develop benchmark methods that extend the token-based
language model to consider layout features like humans. Empirical results show
that layout is critical for VRD-based extraction, and system demonstration also
verifies that the extracted knowledge can help locate the answers that users
care about.

该研究构建了基于文档的对话系统，旨在从视觉丰富的文档（VRD）中提取结构和语义知识，以生成准确的回复，为此创建了一个布局感知的文档级信息提取数据集 LIE，并开发了基准方法考虑人类的布局特征，实验结果表明布局对基于 VRD 的提取至关重要，系统演示也验证了提取的知识能够帮助用户找到关心的答案。LIE 包含来自产品和官方文档中 4,061 页的 62k 注释，成为我们所知最大的基于 VRD 的信息提取数据集。