We present a comprehensive study of chart visual question-answering(QA) task,
to address the challenges faced in comprehending and extracting data from chart
visualizations within documents. Despite efforts to tackle this problem using
synthetic charts, solutions are limited by the shortage of annotated real-world
data. To fill this gap, we introduce a benchmark and dataset for chart visual
QA on real-world charts, offering a systematic analysis of the task and a novel
taxonomy for template-based chart question creation. Our contribution includes
the introduction of a new answer type, 'list', with both ranked and unranked
variations. Our study is conducted on a real-world chart dataset from
scientific literature, showcasing higher visual complexity compared to other
works. Our focus is on template-based QA and how it can serve as a standard for
evaluating the first-order logic capabilities of models. The results of our
experiments, conducted on a real-world out-of-distribution dataset, provide a
robust evaluation of large-scale pre-trained models and advance the field of
chart visual QA and formal logic verification for neural networks in general.

我们在真实世界的图表中对图表视觉问答任务进行了全面的研究，提出了一种基于模板的图表问题创建的新型分类法，并引入了一个包含新答案类型的数据集，从而为模型的一阶逻辑能力评估提供了一个标准，该研究通过对真实世界超出分布的数据集的实验结果，对大规模预训练模型进行了稳健评估，并推进了图表视觉问答和神经网络的形式逻辑验证领域。