In this paper, we explore a forward-thinking question: Is GPT-4V effective at
low-level data analysis tasks on charts? To this end, we first curate a
large-scale dataset, named ChartInsights, consisting of 89,388 quartets (chart,
task, question, answer) and covering 10 widely-used low-level data analysis
tasks on 7 chart types. Firstly, we conduct systematic evaluations to
understand the capabilities and limitations of 18 advanced MLLMs, which include
12 open-source models and 6 closed-source models. Starting with a standard
textual prompt approach, the average accuracy rate across the 18 MLLMs is
36.17%. Among all the models, GPT-4V achieves the highest accuracy, reaching
56.13%. To understand the limitations of multimodal large models in low-level
data analysis tasks, we have designed various experiments to conduct an
in-depth test of capabilities of GPT-4V. We further investigate how visual
modifications to charts, such as altering visual elements (e.g. changing color
schemes) and introducing perturbations (e.g. adding image noise), affect
performance of GPT-4V. Secondly, we present 12 experimental findings. These
findings suggest potential of GPT-4V to revolutionize interaction with charts
and uncover the gap between human analytic needs and capabilities of GPT-4V.
Thirdly, we propose a novel textual prompt strategy, named Chain-of-Charts,
tailored for low-level analysis tasks, which boosts model performance by
24.36%, resulting in an accuracy of 80.49%. Furthermore, by incorporating a
visual prompt strategy that directs attention of GPT-4V to question-relevant
visual elements, we further improve accuracy to 83.83%. Our study not only
sheds light on the capabilities and limitations of GPT-4V in low-level data
analysis tasks but also offers valuable insights for future research.

我们研究了 GPT-4V 在图表的低级数据分析任务中的有效性，并通过实验结果发现其潜力和局限性，同时提出了一种名为 “Chain-of-Charts” 的新颖文本提示策略，能够提高模型性能并揭示人类分析需求与 GPT-4V 能力之间的差距。

评估基于任务的多语言语言模型对图表的有效性

Evaluating Task-based Effectiveness of MLLMs on Charts

There is evidence that address matching plays a crucial role in many areas
such as express delivery, online shopping and so on. Address has a hierarchical
structure, in contrast to unstructured texts, which can contribute valuable
information for address matching. Based on this idea, this paper proposes a
novel method to leverage the hierarchical information in deep learning method
that not only improves the ability of existing methods to handle irregular
address, but also can pay closer attention to the special part of address.
Experimental findings demonstrate that the proposed method improves the current
approach by 3.2% points.

本文提出了一种基于深度学习的层次化信息利用方法，可以更好地解决不规则地址匹配的问题并更加关注地址中的特殊部分，实验结果表明，该方法可以将现有方法的准确率提高 3.2%。

基于层次信息的地址匹配

Address Matching Based On Hierarchical Information

Social dilemmas are situations in which collective interests are at odds with
private interests: pollution, depletion of natural resources, and intergroup
conflicts, are at their core social dilemmas.
Because of their multidisciplinarity and their importance, social dilemmas
have been studied by economists, biologists, psychologists, sociologists, and
political scientists. These studies typically explain tendency to cooperation
by dividing people in proself and prosocial types, or appealing to forms of
external control or, in iterated social dilemmas, to long-term strategies.
But recent experiments have shown that cooperation is possible even in
one-shot social dilemmas without forms of external control and the rate of
cooperation typically depends on the payoffs. This makes impossible a
predictive division between proself and prosocial people and proves that people
have attitude to cooperation by nature.
The key innovation of this article is in fact to postulate that humans have
attitude to cooperation by nature and consequently they do not act a priori as
single agents, as assumed by standard economic models, but they forecast how a
social dilemma would evolve if they formed coalitions and then they act
according to their most optimistic forecast. Formalizing this idea we propose
the first predictive model of human cooperation able to organize a number of
different experimental findings that are not explained by the standard model.
We show also that the model makes satisfactorily accurate quantitative
predictions of population average behavior in one-shot social dilemmas.

本文研究社会困境中人们的合作行为，提出人类天生具有合作的倾向，并建立了第一个能够预测人类合作行为的模型，该模型考虑到人们能够形成联盟来预测社会困境的演化，并根据最乐观的预测来行动。实验结果表明，即使在没有外部控制的情况下，人们也能够在一次性的社会困境中合作。