Document image classification is different from plain-text document
classification and consists of classifying a document by understanding the
content and structure of documents such as forms, emails, and other such
documents. We show that the only existing dataset for this task (Lewis et al.,
2006) has several limitations and we introduce two newly curated multilingual
datasets WIKI-DOC and MULTIEURLEX-DOC that overcome these limitations. We
further undertake a comprehensive study of popular visually-rich document
understanding or Document AI models in previously untested setting in document
image classification such as 1) multi-label classification, and 2) zero-shot
cross-lingual transfer setup. Experimental results show limitations of
multilingual Document AI models on cross-lingual transfer across typologically
distant languages. Our datasets and findings open the door for future research
into improving Document AI models.

文档图像分类是一项与纯文本文档分类不同的任务，它通过理解表单、电子邮件和其他文档的内容和结构来对文档进行分类。本研究介绍了两个新的多语言数据集 WIKI-DOC 和 MULTIEURLEX-DOC，克服了现有数据集的限制。此外，本研究对以前未经测试的文档图像分类任务中的流行视觉丰富的文档理解或文档智能模型进行了全面研究，包括多标签分类和零样本跨语言迁移设置。实验结果显示多语言文档智能模型在跨语言迁移上存在一定的局限性。我们的数据集和研究结果为未来改进文档智能模型打开了大门。

一个多模态多语言文件图像分类基准

A Multi-Modal Multilingual Benchmark for Document Image Classification

The use of visually-rich documents (VRDs) in various fields has created a
demand for Document AI models that can read and comprehend documents like
humans, which requires the overcoming of technical, linguistic, and cognitive
barriers. Unfortunately, the lack of appropriate datasets has significantly
hindered advancements in the field. To address this issue, we introduce
\textsc{DocTrack}, a VRD dataset really aligned with human eye-movement
information using eye-tracking technology. This dataset can be used to
investigate the challenges mentioned above. Additionally, we explore the impact
of human reading order on document understanding tasks and examine what would
happen if a machine reads in the same order as a human. Our results suggest
that although Document AI models have made significant progress, they still
have a long way to go before they can read VRDs as accurately, continuously,
and flexibly as humans do. These findings have potential implications for
future research and development of Document AI models. The data is available at
https://github.com/hint-lab/doctrack.

使用者眼动追踪技术对齐的 VRD 数据集，研究人类阅读顺序对文档理解任务的影响，结果显示虽然文档 AI 模型取得了显著进展，但在与人类相比准确、连续和灵活地阅读 VRD 方面仍有很长的路要走，对未来的文档 AI 模型研究和开发具有潜在的影响。