Text role classification involves classifying the semantic role of textual
elements within scientific charts. For this task, we propose to finetune two
pretrained multimodal document layout analysis models, LayoutLMv3 and UDOP, on
chart datasets. The transformers utilize the three modalities of text, image,
and layout as input. We further investigate whether data augmentation and
balancing methods help the performance of the models. The models are evaluated
on various chart datasets, and results show that LayoutLMv3 outperforms UDOP in
all experiments. LayoutLMv3 achieves the highest F1-macro score of 82.87 on the
ICPR22 test dataset, beating the best-performing model from the ICPR22
CHART-Infographics challenge. Moreover, the robustness of the models is tested
on a synthetic noisy dataset ICPR22-N. Finally, the generalizability of the
models is evaluated on three chart datasets, CHIME-R, DeGruyter, and EconBiz,
for which we added labels for the text roles. Findings indicate that even in
cases where there is limited training data, transformers can be used with the
help of data augmentation and balancing methods. The source code and datasets
are available on GitHub under
this https URL

文本角色分类涉及对科学图表中的文本元素进行语义角色分类。我们提出在图表数据集上对两个预训练的多模态文档布局分析模型 LayoutLMv3 和 UDOP 进行微调，并利用文本、图像和布局这三种模态作为输入。我们进一步研究了数据增强和平衡方法是否对模型的性能有帮助。模型在各种图表数据集上进行评估，结果表明 LayoutLMv3 在所有实验中表现优于 UDOP。LayoutLMv3 在 ICPR22 测试数据集上获得了 82.87 的最高 F1 宏分数，在 ICPR22 CHART-Infographics 挑战中超过了最佳模型。此外，模型的鲁棒性在合成噪声数据集 ICPR22-N 上进行了测试。最后，我们评估了模型在三个带有文本角色标签的图表数据集 CHIME-R、DeGruyter 和 EconBiz 上的泛化能力。研究结果表明，即使在训练数据有限的情况下，通过数据增强和平衡方法可以使用 transformers。源代码和数据集可在 GitHub 上的此网址找到。