This paper introduces DONUT-hole, a sparse OCR-free visual document
understanding (VDU) model that addresses the limitations of its predecessor
model, dubbed DONUT. The DONUT model, leveraging a transformer architecture,
overcoming the challenges of separate optical character recognition (OCR) and
visual semantic understanding (VSU) components. However, its deployment in
production environments and edge devices is hindered by high memory and
computational demands, particularly in large-scale request services. To
overcome these challenges, we propose an optimization strategy based on
knowledge distillation and model pruning. Our paradigm to produce DONUT-hole,
reduces the model denisty by 54\% while preserving performance. We also achieve
a global representational similarity index between DONUT and DONUT-hole based
on centered kernel alignment (CKA) metric of 0.79. Moreover, we evaluate the
effectiveness of DONUT-hole in the document image key information extraction
(KIE) task, highlighting its potential for developing more efficient VDU
systems for logistic companies.

DONUT-hole 是一种稀疏 OCR-free 视觉文档理解（VDU）模型，通过优化策略实现了模型密度的 54% 减少，同时保持性能，并在文档图像关键信息提取（KIE）任务中展示其对物流公司开发更高效 VDU 系统的潜力。