Chart summarization is a crucial task for blind and visually impaired
individuals as it is their primary means of accessing and interpreting
graphical data. Crafting high-quality descriptions is challenging because it
requires precise communication of essential details within the chart without
vision perception. Many chart analysis methods, however, produce brief,
unstructured responses that may contain significant hallucinations, affecting
their reliability for blind people. To address these challenges, this work
presents three key contributions: (1) We introduce the AltChart dataset,
comprising 10,000 real chart images, each paired with a comprehensive summary
that features long-context, and semantically rich annotations. (2) We propose a
new method for pretraining Vision-Language Models (VLMs) to learn fine-grained
chart representations through training with multiple pretext tasks, yielding a
performance gain with ${\sim}2.5\%$. (3) We conduct extensive evaluations of
four leading chart summarization models, analyzing how accessible their
descriptions are. Our dataset and codes are publicly available on our project
page: this https URL

图表概括对于盲人和视障人士来说是一项至关重要的任务，因为它是他们获取和解释图形数据的主要手段。本研究提出了三个关键贡献：引入了 AltChart 数据集，提出了一个新的预训练视觉语言模型的方法，以及对四种主流图表概括模型进行了全面评估。

AltChart: 多预训练任务提升基于 VLM 的图表摘要

AltChart: Enhancing VLM-based Chart Summarization Through Multi-Pretext  Tasks

Data visualization serves as a critical means for presenting data and mining
its valuable insights. The task of chart summarization, through natural
language processing techniques, facilitates in-depth data analysis of charts.
However, there still are notable deficiencies in terms of visual-language
matching and reasoning ability for existing approaches. To address these
limitations, this study constructs a large-scale dataset of comprehensive
chart-caption pairs and fine-tuning instructions on each chart. Thanks to the
broad coverage of various topics and visual styles within this dataset, better
matching degree can be achieved from the view of training data. Moreover, we
propose an innovative chart summarization method, ChartThinker, which
synthesizes deep analysis based on chains of thought and strategies of context
retrieval, aiming to improve the logical coherence and accuracy of the
generated summaries. Built upon the curated datasets, our trained model
consistently exhibits superior performance in chart summarization tasks,
surpassing 8 state-of-the-art models over 7 evaluation metrics. Our dataset and
codes are publicly accessible.

数据可视化是呈现数据和挖掘其有价值洞见的关键手段。通过自然语言处理技术，图表总结任务促进了对图表的深入数据分析。本研究构建了一个大规模的全面图表 - 标题数据集，提供了每个图表的微调指导，以解决现有方法在视觉 - 语言匹配和推理能力方面存在的明显不足。基于这个数据集的广泛覆盖范围，可以实现更好的训练数据视角匹配度。此外，我们提出了一种创新的图表总结方法 ChartThinker，它基于思维链和上下文检索策略合成了深层分析，旨在提高生成摘要的逻辑连贯性和准确性。在精心策划的数据集基础上，我们训练的模型在图表总结任务中始终表现出优越性能，在 7 个评估指标上超过了 8 种最先进的模型。我们的数据集和代码可公开访问。