Despite the success of large language models (LLMs) in Text-to-SQL tasks, open-source LLMs encounter challenges in contextual understanding and response coherence. To tackle these issues, we present \ours, a systematic methodology tailored for Text-to-SQL with open-source LLMs. Our contributions include a comprehensive evaluation of open-source LLMs in Text-to-SQL tasks, the \openprompt strategy for effective question representation, and novel strategies for supervised fine-tuning. We explore the benefits of Chain-of-Thought in step-by-step inference and propose the \openexample method for enhanced few-shot learning. Additionally, we introduce token-efficient techniques, such as \textbf{Variable-length Open DB Schema}, \textbf{Target Column Truncation}, and \textbf{Example Column Truncation}, addressing challenges in large-scale databases. Our findings emphasize the need for further investigation into the impact of supervised fine-tuning on contextual learning capabilities. Remarkably, our method significantly improved Llama2-7B from 2.54\% to 41.04\% and Code Llama-7B from 14.54\% to 48.24\% on the BIRD-Dev dataset. Notably, the performance of Code Llama-7B surpassed GPT-4 (46.35\%) on the BIRD-Dev dataset.

我们提出了一种针对开源大型语言模型在Text-to-SQL任务中的上下文理解和响应连贯性问题的系统方法，包括对开源大型语言模型在Text-to-SQL任务中的全面评估，以及用于有效问题表示的openprompt策略、监督微调的新策略、步骤推理中Chain-of-Thought的优势探索和用于增强少样本学习的openexample方法。此外，我们引入了一些高效的技术，如可变长度的开发数据库架构、目标列截断和示例列截断，解决了大规模数据库中的挑战。我们的研究结果强调了对监督微调对上下文学习能力的影响进行进一步研究的必要性。值得注意的是，我们的方法显著提高了Llama2-7B在BIRD-Dev数据集上的性能，从2.54%提高到41.04%，Code Llama-7B在BIRD-Dev数据集上的性能甚至超过了GPT-4（46.35%）。

Open-SQL 框架：在开源大型语言模型上增强文本到SQL转换