A number of tasks have been proposed recently to facilitate easy access to
charts such as chart QA and summarization. The dominant paradigm to solve these
tasks has been to fine-tune a pretrained model on the task data. However, this
approach is not only expensive but also not generalizable to unseen tasks. On
the other hand, large language models (LLMs) have shown impressive
generalization capabilities to unseen tasks with zero- or few-shot prompting.
However, their application to chart-related tasks is not trivial as these tasks
typically involve considering not only the underlying data but also the visual
features in the chart image. We propose PromptChart, a multimodal few-shot
prompting framework with LLMs for chart-related applications. By analyzing the
tasks carefully, we have come up with a set of prompting guidelines for each
task to elicit the best few-shot performance from LLMs. We further propose a
strategy to inject visual information into the prompts. Our experiments on
three different chart-related information consumption tasks show that with
properly designed prompts LLMs can excel on the benchmarks, achieving
state-of-the-art.

通过分析任务，我们提出了用于图表相关应用的多模态少样本提示框架 PromptChart，为每个任务制定了一套提示指南，以从大型语言模型中提取出最好的少样本性能，并提出了一种将视觉信息注入到提示中的策略。我们在三个不同的图表相关信息消费任务上进行的实验表明，通过合理设计的提示，大型语言模型在基准测试中表现出色，达到了最先进的水平。