Large Language Models (LLMs) have issues with document question answering
(QA) in situations where the document is unable to fit in the small context
length of an LLM. To overcome this issue, most existing works focus on
retrieving the relevant context from the document, representing them as plain
text. However, documents such as PDFs, web pages, and presentations are
naturally structured with different pages, tables, sections, and so on.
Representing such structured documents as plain text is incongruous with the
user's mental model of these documents with rich structure. When a system has
to query the document for context, this incongruity is brought to the fore, and
seemingly trivial questions can trip up the QA system. To bridge this
fundamental gap in handling structured documents, we propose an approach called
PDFTriage that enables models to retrieve the context based on either structure
or content. Our experiments demonstrate the effectiveness of the proposed
PDFTriage-augmented models across several classes of questions where existing
retrieval-augmented LLMs fail. To facilitate further research on this
fundamental problem, we release our benchmark dataset consisting of 900+
human-generated questions over 80 structured documents from 10 different
categories of question types for document QA.

提供了针对结构化文档的文档问答技术 PDFTriage，通过基于结构或内容的检索，解决了大型语言模型在处理结构化文档时的困境，并发布了包含 900 多个人工生成问题的基准数据集。

PDFTriage：长文结构化文档上的问答

PDFTriage: Question Answering over Long, Structured Documents

In e-commerce search, relevance between query and documents is an essential
requirement for satisfying user experience. Different from traditional
e-commerce platforms that offer products, users search on life service
platforms such as Meituan mainly for product providers, which usually have
abundant structured information, e.g. name, address, category, thousands of
products. Modeling search relevance with these rich structured contents is
challenging due to the following issues: (1) there is language distribution
discrepancy among different fields of structured document, making it difficult
to directly adopt off-the-shelf pretrained language model based methods like
BERT. (2) different fields usually have different importance and their length
vary greatly, making it difficult to extract document information helpful for
relevance matching.
To tackle these issues, in this paper we propose a novel two-stage
pretraining and matching architecture for relevance matching with rich
structured documents. At pretraining stage, we propose an effective pretraining
method that employs both query and multiple fields of document as inputs,
including an effective information compression method for lengthy fields. At
relevance matching stage, a novel matching method is proposed by leveraging
domain knowledge in search query to generate more effective document
representations for relevance scoring. Extensive offline experiments and online
A/B tests on millions of users verify that the proposed architectures
effectively improve the performance of relevance modeling. The model has
already been deployed online, serving the search traffic of Meituan for over a
year.

在电子商务搜索中，通过使用丰富的结构化信息和两阶段的预训练和匹配架构，可以有效改进相关性建模的性能。

SPM: 美团搜索中结构化预训练和匹配架构的相关性建模

SPM: Structured Pretraining and Matching Architectures for Relevance  Modeling in Meituan Search

We introduce a novel type of text representation that preserves the 2D layout
of a document. This is achieved by encoding each document page as a
two-dimensional grid of characters. Based on this representation, we present a
generic document understanding pipeline for structured documents. This pipeline
makes use of a fully convolutional encoder-decoder network that predicts a
segmentation mask and bounding boxes. We demonstrate its capabilities on an
information extraction task from invoices and show that it significantly
outperforms approaches based on sequential text or document images.

本文介绍一种新颖的文本表示方法，该方法保留文档的 2D 布局，通过将每个文档页面编码为字符的二维网格来实现。基于此表示，我们提出了一个结构化文档的通用文档理解流程。该流程利用完全卷积编码器 - 解码器网络预测分割掩模和边界框。我们在发票信息提取任务上展示了其功效，并展示它在超过基于顺序文本或文档图像的方法中的显着表现优势。