Detecting structural similarity between queries is essential for selecting
examples in in-context learning models. However, assessing structural
similarity based solely on the natural language expressions of queries, without
considering SQL queries, presents a significant challenge. This paper explores
the significance of this similarity metric and proposes a model for accurately
estimating it. To achieve this, we leverage a dataset comprising 170k question
pairs, meticulously curated to train a similarity prediction model. Our
comprehensive evaluation demonstrates that the proposed model adeptly captures
the structural similarity between questions, as evidenced by improvements in
Kendall-Tau distance and precision@k metrics. Notably, our model outperforms
strong competitive embedding models from OpenAI and Cohere. Furthermore,
compared to these competitive models, our proposed encoder enhances the
downstream performance of NL2SQL models in 1-shot in-context learning scenarios
by 1-2\% for GPT-3.5-turbo, 4-8\% for CodeLlama-7B, and 2-3\% for
CodeLlama-13B.

本文研究了在上下文学习模型中选择示例的结构相似性检测，并提出了一种准确估计这种相似性的模型，通过在训练语义预测模型的精心策划的 170k 个问题对中利用基于 SQL 查询的相似性度量。全面的评估结果表明，所提出的模型在捕捉问题间结构相似性方面表现出色，表现体现在 Kendall-Tau 距离和 precision@k 指标的改进上。值得注意的是，与 OpenAI 和 Cohere 等竞争性嵌入模型相比，我们的模型在 1-shot 上下文学习场景中提高了 GPT-3.5-turbo、CodeLlama-7B 和 CodeLlama-13B 的 NL2SQL 模型表现 1-2%、4-8% 和 2-3%。