Large language models (LLMs) exhibit remarkable performance improvement
through in-context learning (ICL) by leveraging task-specific examples in the
input. However, the mechanisms behind this improvement remain elusive. In this
work, we investigate how LLM embeddings and attention representations change
following in-context-learning, and how these changes mediate improvement in
behavior. We employ neuroscience-inspired techniques such as representational
similarity analysis (RSA) and propose novel methods for parameterized probing
and measuring ratio of attention to relevant vs. irrelevant information in
Llama-2 70B and Vicuna 13B. We designed three tasks with a priori relationships
among their conditions: reading comprehension, linear regression, and
adversarial prompt injection. We formed hypotheses about expected similarities
in task representations to investigate latent changes in embeddings and
attention. Our analyses revealed a meaningful correlation between changes in
both embeddings and attention representations with improvements in behavioral
performance after ICL. This empirical framework empowers a nuanced
understanding of how latent representations affect LLM behavior with and
without ICL, offering valuable tools and insights for future research and
practical applications.

大型语言模型通过背景学习在任务导向的例子中取得了显著的性能提升。本文研究了背景学习对语言模型嵌入和注意力表示的影响，以及这些变化如何对行为改进起到中介作用。通过神经科学启发的技术，我们发现在背景学习后，嵌入和注意力表示的变化与行为性能的改善之间存在有意义的相关性，这为对 LLM 行为的潜在表示进行细致理解提供了有价值的工具和见解。