This study investigates various approaches to using Large Language Models
(LLMs) for Text-to-SQL program synthesis, focusing on the outcomes and insights
derived. Employing the popular Text-to-SQL dataset, spider, the goal was to
input a natural language question along with the database schema and output the
correct SQL SELECT query. The initial approach was to fine-tune a local and
open-source model to generate the SELECT query. After QLoRa fine-tuning
WizardLM's WizardCoder-15B model on the spider dataset, the execution accuracy
for generated queries rose to a high of 61%. With the second approach, using
the fine-tuned gpt-3.5-turbo-16k (Few-shot) + gpt-4-turbo (Zero-shot error
correction), the execution accuracy reached a high of 82.1%. Of all the
incorrect queries, most can be categorized into a seven different categories of
what went wrong: selecting the wrong columns or wrong order of columns,
grouping by the wrong column, predicting the wrong values in conditionals,
using different aggregates than the ground truth, extra or too few JOIN
clauses, inconsistencies in the Spider dataset, and lastly completely incorrect
query structure. Most if not all of the queries fall into these categories and
it is insightful to understanding where the faults still lie with LLM program
synthesis and where they can be improved.

该研究通过使用大型语言模型（LLMs）来进行文本到 SQL 程序合成的各种方法以及相关的结果和见解，通过使用流行的 Text-to-SQL 数据集（spider）输入自然语言问题和数据库模式并生成正确的 SQL SELECT 查询。通过细调 WizardLM 的 WizardCoder-15B 模型和 fine-tuning gpt-3.5-turbo-16k（Few-shot）+ gpt-4-turbo（Zero-shot error correction）的方式，查询的执行准确率达到了较高的 82.1%。大部分错误查询可以归为七个不同的类别，这揭示了 LLM 程序合成的瑕疵以及可改进的方向。