In the field of document understanding, significant advances have been made
in the fine-tuning of Multimodal Large Language Models (MLLMs) with
instruction-following data. Nevertheless, the potential of text-grounding
capability within text-rich scenarios remains underexplored. In this paper, we
present a text-grounding document understanding model, termed TGDoc, which
addresses this deficiency by enhancing MLLMs with the ability to discern the
spatial positioning of text within images. Empirical evidence suggests that
text-grounding improves the model's interpretation of textual content, thereby
elevating its proficiency in comprehending text-rich images. Specifically, we
compile a dataset containing 99K PowerPoint presentations sourced from the
internet. We formulate instruction tuning tasks including text detection,
recognition, and spotting to facilitate the cohesive alignment between the
visual encoder and large language model. Moreover, we curate a collection of
text-rich images and prompt the text-only GPT-4 to generate 12K high-quality
conversations, featuring textual locations within text-rich scenarios. By
integrating text location data into the instructions, TGDoc is adept at
discerning text locations during the visual question process. Extensive
experiments demonstrate that our method achieves state-of-the-art performance
across multiple text-rich benchmarks, validating the effectiveness of our
method.

在文档理解领域，本文提出了一种文本定位的文档理解模型，命名为 TGDoc，通过增强多模态大型语言模型（MLLMs）的能力来识别图像内文本的空间位置，以提高文本内容解释的准确性，从而提高对文本丰富图像的理解能力。实验证据表明，文本定位方法在多个文本丰富基准测试中取得了最先进的性能，验证了我们方法的有效性。