Current multimodal large language models (MLLMs) face significant challenges
in visual document understanding (VDU) tasks due to the high resolution, dense
text, and complex layouts typical of document images. These characteristics
demand a high level of detail perception ability from MLLMs. While increasing
input resolution improves detail perception, it also leads to longer sequences
of visual tokens, increasing computational costs and straining the models'
ability to handle long contexts. To address these challenges, we introduce
DocKylin, a document-centric MLLM that performs visual content slimming at both
the pixel and token levels, thereby reducing token sequence length in VDU
scenarios. DocKylin utilizes an Adaptive Pixel Slimming (APS) preprocessing
module to perform pixel-level slimming, increasing the proportion of
informative pixels. Moreover, DocKylin incorporates a novel Dynamic Token
Slimming (DTS) module to conduct token-level slimming, filtering essential
tokens and removing others to create a compressed, adaptive visual sequence.
Experiments demonstrate DocKylin's promising performance across various VDU
benchmarks. Notably, both the proposed APS and DTS are parameter-free,
facilitating easy integration into existing MLLMs, and our experiments indicate
their potential for broader applications.

DocKylin 是一种基于文档的多模态大型语言模型，使用自适应像素精简和动态标记精简模块，以降低视觉文档理解任务中的标记序列长度，并在各种评测中展现了良好的性能。