Recently, the advent of Large Visual-Language Models (LVLMs) has received
increasing attention across various domains, particularly in the field of
visual document understanding (VDU). Different from conventional
vision-language tasks, VDU is specifically concerned with text-rich scenarios
containing abundant document elements. Nevertheless, the importance of
fine-grained features remains largely unexplored within the community of LVLMs,
leading to suboptimal performance in text-rich scenarios. In this paper, we
abbreviate it as the fine-grained feature collapse issue. With the aim of
filling this gap, we propose a contrastive learning framework, termed Document
Object COntrastive learning (DoCo), specifically tailored for the downstream
tasks of VDU. DoCo leverages an auxiliary multimodal encoder to obtain the
features of document objects and align them to the visual features generated by
the vision encoder of LVLM, which enhances visual representation in text-rich
scenarios. It can represent that the contrastive learning between the visual
holistic representations and the multimodal fine-grained features of document
objects can assist the vision encoder in acquiring more effective visual cues,
thereby enhancing the comprehension of text-rich documents in LVLMs. We also
demonstrate that the proposed DoCo serves as a plug-and-play pre-training
method, which can be employed in the pre-training of various LVLMs without
inducing any increase in computational complexity during the inference process.
Extensive experimental results on multiple benchmarks of VDU reveal that LVLMs
equipped with our proposed DoCo can achieve superior performance and mitigate
the gap between VDU and generic vision-language tasks.

利用对比学习框架 DoCo，该研究填补了大型视觉 - 语言模型在处理富文本场景中的细粒度特征缺失问题，提高了对文本丰富的文档的视觉表示，并在多个视觉文档理解基准上取得了优越的性能。