In this paper, we conduct a study to utilize LLMs as a solution for decision
making that requires complex data analysis. We define Decision QA as the task
of answering the best decision, $d_{best}$, for a decision-making question $Q$,
business rules $R$ and a database $D$. Since there is no benchmark that can
examine Decision QA, we propose Decision QA benchmark, DQA. It has two
scenarios, Locating and Building, constructed from two video games (Europa
Universalis IV and Victoria 3) that have almost the same goal as Decision QA.
To address Decision QA effectively, we also propose a new RAG technique called
the iterative plan-then-retrieval augmented generation (PlanRAG). Our
PlanRAG-based LM generates the plan for decision making as the first step, and
the retriever generates the queries for data analysis as the second step. The
proposed method outperforms the state-of-the-art iterative RAG method by 15.8%
in the Locating scenario and by 7.4% in the Building scenario, respectively. We
release our code and benchmark at this https URL

利用 LLMs 作为复杂数据分析决策的解决方案，我们定义了 Decision QA 任务，在给定决策问题 Q、业务规则 R 和数据库 D 的情况下，回答最佳决策 $d_{best}$ 的任务。为了有效应对 Decision QA，我们提出了一种名为 PlanRAG 的新型技术，通过迭代的规划 - 检索增强生成方法，在 Locating 和 Building 两个场景中胜过先进的迭代 RAG 方法。