This paper proposes LayoutLLM, a more flexible document analysis method for
understanding imaged documents. Visually Rich Document Understanding tasks,
such as document image classification and information extraction, have gained
significant attention due to their importance. Existing methods have been
developed to enhance document comprehension by incorporating pre-training
awareness of images, text, and layout structure. However, these methods require
fine-tuning for each task and dataset, and the models are expensive to train
and operate. To overcome this limitation, we propose a new LayoutLLM that
integrates these with large-scale language models (LLMs). By leveraging the
strengths of existing research in document image understanding and LLMs'
superior language understanding capabilities, the proposed model, fine-tuned
with multimodal instruction datasets, performs an understanding of document
images in a single model. Our experiments demonstrate improvement over the
baseline model in various document analysis tasks.

本文提出了 LayoutLLM，一种更灵活的文档分析方法，用于理解图像化文档。通过利用现有研究在文档图像理解和大规模语言模型的优势，通过与多模态指令数据集进行微调，提出的模型在单个模型中执行对文档图像的理解，并通过实验证明在各种文档分析任务中改进了基线模型。