In-Context learning is the paradigm that adapts large language models to
downstream tasks by providing a few examples. Few-shot selection -- selecting
appropriate examples for each test instance separately -- is important for
in-context learning. In this paper, we propose Skill-KNN, a skill-based
few-shot selection method for in-context learning. The key advantages of
Skill-KNN include: (1) it addresses the problem that existing methods based on
pre-trained embeddings can be easily biased by surface natural language
features that are not important for the target task; (2) it does not require
training or fine-tuning of any models, making it suitable for frequently
expanding or changing example banks. The key insight is to optimize the inputs
fed into the embedding model, rather than tuning the model itself. Technically,
Skill-KNN generates the skill-based representations for each test case and
candidate example by utilizing a pre-processing few-shot prompting, thus
eliminating unimportant surface features. Experimental results across four
cross-domain semantic parsing tasks and four backbone models show that
Skill-KNN significantly outperforms existing methods.

本研究提出了基于技能的少样本选择方法 Skill-KNN，通过优化输入，生成技能表示法，解决了现有基于预训练嵌入模型的表面自然语言特征易受干扰的问题。经过实验证明，在四个跨领域语义分析任务和四个骨架模型中，Skill-KNN 的表现明显优于现有方法。

基于技能的少样本选择用于上下文学习

Skill-Based Few-Shot Selection for In-Context Learning

We present SParC, a dataset for cross-domainSemanticParsing inContext that
consists of 4,298 coherent question sequences (12k+ individual questions
annotated with SQL queries). It is obtained from controlled user interactions
with 200 complex databases over 138 domains. We provide an in-depth analysis of
SParC and show that it introduces new challenges compared to existing datasets.
SParC demonstrates complex contextual dependencies, (2) has greater semantic
diversity, and (3) requires generalization to unseen domains due to its
cross-domain nature and the unseen databases at test time. We experiment with
two state-of-the-art text-to-SQL models adapted to the context-dependent,
cross-domain setup. The best model obtains an exact match accuracy of 20.2%
over all questions and less than10% over all interaction sequences, indicating
that the cross-domain setting and the con-textual phenomena of the dataset
present significant challenges for future research. The dataset, baselines, and
leaderboard are released at this https URL

本研究提出了 SParC 数据集，旨在解决复杂的语义解析问题，该数据集跨域，包含了超过 138 个领域的 200 个复杂数据库，包括 4298 个连贯的问题序列和 12000 多个与 SQL 查询一起注释的单独问题。该数据集引入了新的挑战，如复杂的上下文依赖性、更大的语义多样性，需要在未见领域中进行泛化。研究者使用两种最先进的文本到 SQL 模型进行实验。最佳模型在所有问题上的完全匹配准确率为 20.2％，在所有交互序列上少于 10％，这表明数据集的跨域设置和上下文现象对未来研究具有重大挑战。