Large language models (LMs) are able to in-context learn -- perform a new
task via inference alone by conditioning on a few input-label pairs
(demonstrations) and making predictions for new inputs. However, there has been
little understanding of how the model learns and which aspects of the
demonstrations contribute to end task performance. In this paper, we show that
ground truth demonstrations are in fact not required -- randomly replacing
labels in the demonstrations barely hurts performance on a range of
classification and multi-choce tasks, consistently over 12 different models
including GPT-3. Instead, we find that other aspects of the demonstrations are
the key drivers of end task performance, including the fact that they provide a
few examples of (1) the label space, (2) the distribution of the input text,
and (3) the overall format of the sequence. Together, our analysis provides a
new way of understanding how and why in-context learning works, while opening
up new questions about how much can be learned from large language models
through inference alone.

本研究分析显示：大型语言模型不需要准确的演示，而是通过演示提供的标签空间、输入文本的分布和序列的整体格式等方面驱动任务表现的提高。因此，揭示了语境学习的原理和作用方式，同时提出了新的问题，即能否仅仅通过推理来学习大型语言模型的更多内容。