Language Models (LMs) can perform new tasks by adapting to a few in-context
examples. For humans, explanations that connect examples to task principles can
improve learning. We therefore investigate whether explanations of few-shot
examples can help LMs. We annotate questions from 40 challenging tasks with
answer explanations, and various matched control explanations. We evaluate how
different types of explanations, instructions, and controls affect zero- and
few-shot performance. We analyze these results using statistical multilevel
modeling techniques that account for the nested dependencies among conditions,
tasks, prompts, and models. We find that explanations can improve performance
-- even without tuning. Furthermore, explanations hand-tuned for performance on
a small validation set offer substantially larger benefits, and building a
prompt by selecting examples and explanations together substantially improves
performance over selecting examples alone. Finally, even untuned explanations
outperform carefully matched controls, suggesting that the benefits are due to
the link between an example and its explanation, rather than lower-level
features. However, only large models benefit. In summary, explanations can
support the in-context learning of large LMs on challenging tasks.

通过对 40 个具有挑战性的任务提出带有答案解释的问题及各种匹配控制解释，我们发现解释可以提高大型语言模型在具有挑战性的任务上的零次和少量样本的表现，并且仅大型模型有益处。