Large language models (LLMs) have emerged as a new paradigm for Text-to-SQL
task. However, the absence of a systematical benchmark inhibits the development
of designing effective, efficient and economic LLM-based Text-to-SQL solutions.
To address this challenge, in this paper, we first conduct a systematical and
extensive comparison over existing prompt engineering methods, including
question representation, example selection and example organization, and with
these experimental results, we elaborates their pros and cons. Based on these
findings, we propose a new integrated solution, named DAIL-SQL, which refreshes
the Spider leaderboard with 86.6% execution accuracy and sets a new bar.
Towards an efficient and economic LLM-based Text-to-SQL solution, we emphasize
the token efficiency in prompt engineering and compare the prior studies under
this metric. Additionally, we investigate open-source LLMs in in-context
learning, and further enhance their performance with task-specific supervised
fine-tuning. Our explorations highlight open-source LLMs' potential in
Text-to-SQL, as well as the advantages and disadvantages of the task-specific
supervised fine-tuning. We hope that our work provides a deeper understanding
of Text-to-SQL with LLMs, and inspire further investigations and broad
applications.

本文对大型语言模型（LLMs）在 Text-to-SQL 任务中的应用进行了研究，提出了一种新的集成解决方案 DAIL-SQL，并通过实验证明了其在 Spider 排行榜上取得了 86.6% 的执行准确率。研究重点在于提示工程中的令牌效率和任务特定的监督微调。通过探索开源 LLMs 在 Text-to-SQL 中的潜力，以及任务特定的监督微调的优势和劣势，希望能够对 LLMs 在 Text-to-SQL 领域提供更深入的了解，并激发进一步的研究和广泛应用。