In-context learning (ICL) has become one of the most popular learning paradigms. While there is a growing body of literature focusing on prompt engineering, there is a lack of systematic analysis comparing the effects of prompts across different models and tasks. To address this gap, we present a comprehensive prompt analysis based on the sensitivity of a function. Our analysis reveals that sensitivity is an unsupervised proxy for model performance, as it exhibits a strong negative correlation with accuracy. We use gradient-based saliency scores to empirically demonstrate how different prompts affect the relevance of input tokens to the output, resulting in different levels of sensitivity. Furthermore, we introduce sensitivity-aware decoding which incorporates sensitivity estimation as a penalty term in the standard greedy decoding. We show that this approach is particularly helpful when information in the input is scarce. Our work provides a fresh perspective on the analysis of prompts, and contributes to a better understanding of the mechanism of ICL.

基于函数的敏感性进行全面的提示分析，揭示了敏感性作为模型性能的一个非监督代理，与准确度呈强负相关。通过梯度敏感度分数，实证展示了不同提示对输出的输入符号的相关性产生不同敏感度。还引入了敏感度感知解码，将敏感度估计作为标准贪婪解码中的惩罚项，特别适用于输入信息稀缺的情况。对提示的分析提供了新的视角，有助于更好地理解ICL机制。

提示在敏感性方面的差异如何？