We present an extensible method for identifying semantic points to reverse
engineer (i.e. extract the values of) data charts, particularly those in
scientific articles. Our method uses a point proposal network (akin to region
proposal networks for object detection) to directly predict the position of
points of interest in a chart, and it is readily extensible to multiple chart
types and chart elements. We focus on complex bar charts in the scientific
literature, on which our model is able to detect salient points with an
accuracy of 0.8705 F1 (@1.5-cell max deviation); it achieves 0.9810 F1 on
synthetically-generated charts similar to those used in prior works. We also
explore training exclusively on synthetic data with novel augmentations,
reaching surprisingly competent performance in this way (0.6621 F1) on real
charts with widely varying appearance, and we further demonstrate our unchanged
method applied directly to synthetic pie charts (0.8343 F1). Datasets, trained
models, and evaluation code are available at
this https URL

通过使用点建议网络，我们提出了一种可扩展的方法来识别语义点，以便从科学文章中反向工程（提取值），特别是数据图表。我们的模型能够准确检测复杂的条形图中的显著点，并且在合成的图表上也表现出很好的性能。

一种可扩展的基于点的数据图表值检测方法

An extensible point-based method for data chart value detection

Infographics are often an integral component of scientific documents for
reporting qualitative or quantitative findings as they make it much simpler to
comprehend the underlying complex information. However, their interpretation
continues to be a challenge for the blind, low-vision, and other print-impaired
(BLV) individuals. In this paper, we propose ChartParser, a fully automated
pipeline that leverages deep learning, OCR, and image processing techniques to
extract all figures from a research paper, classify them into various chart
categories (bar chart, line chart, etc.) and obtain relevant information from
them, specifically bar charts (including horizontal, vertical, stacked
horizontal and stacked vertical charts) which already have several exciting
challenges. Finally, we present the retrieved content in a tabular format that
is screen-reader friendly and accessible to the BLV users. We present a
thorough evaluation of our approach by applying our pipeline to sample
real-world annotated bar charts from research papers.

本文介绍了一种利用深度学习、OCR 和图像处理技术的自动化流程 ChartParser，可以将研究论文中的图表提取出来，进行分类，提取有用信息，并以易于使用的表格格式呈现给盲人、低视力和其他印刷受限 (BLV) 的人群。我们还对该方法进行了详细的评估。

ChartParser：面向印刷障碍者的自动图表解析

ChartParser: Automatic Chart Parsing for Print-Impaired

Bar charts are an effective way to convey numeric information, but today's
algorithms cannot parse them. Existing methods fail when faced with even minor
variations in appearance. Here, we present DVQA, a dataset that tests many
aspects of bar chart understanding in a question answering framework. Unlike
visual question answering (VQA), DVQA requires processing words and answers
that are unique to a particular bar chart. State-of-the-art VQA algorithms
perform poorly on DVQA, and we propose two strong baselines that perform
considerably better. Our work will enable algorithms to automatically extract
numeric and semantic information from vast quantities of bar charts found in
scientific publications, Internet articles, business reports, and many other
areas.

DVQA 数据集是一个检测条形图理解能力的问答框架，与 VQA 相比，我们的方法可以处理特定条形图中独特的问题和答案，研究提出了两个高效的基线模型，其能够从科学出版物、互联网文章、商业报告等多种领域中自动提取数值和语义信息，这项工作将使算法能够自动从大量的条形图中提取信息。