Modern generative models demonstrate impressive capabilities, likely stemming
from an ability to identify and manipulate abstract concepts underlying their
training data. However, fundamental questions remain: what determines the
concepts a model learns, the order in which it learns them, and its ability to
manipulate those concepts? To address these questions, we propose analyzing a
model's learning dynamics via a framework we call the concept space, where each
axis represents an independent concept underlying the data generating process.
By characterizing learning dynamics in this space, we identify how the speed at
which a concept is learned, and hence the order of concept learning, is
controlled by properties of the data we term concept signal. Further, we
observe moments of sudden turns in the direction of a model's learning dynamics
in concept space. Surprisingly, these points precisely correspond to the
emergence of hidden capabilities, i.e., where latent interventions show the
model possesses the capability to manipulate a concept, but these capabilities
cannot yet be elicited via naive input prompting. While our results focus on
synthetically defined toy datasets, we hypothesize a general claim on emergence
of hidden capabilities may hold: generative models possess latent capabilities
that emerge suddenly and consistently during training, though a model might not
exhibit these capabilities under naive input prompting.

通过新的概念空间框架分析模型的学习动态，在合成定义的玩具数据集上发现生成模型具备突然且一致地在训练过程中出现的潜在能力，虽然模型可能在普通输入提示下无法展示这些能力。

隐藏能力的出现：探索概念空间中的学习动态

Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept  Space

We ask whether multilingual language models trained on unbalanced,
English-dominated corpora use English as an internal pivot language -- a
question of key importance for understanding how language models function and
the origins of linguistic bias. Focusing on the Llama-2 family of transformer
models, our study uses carefully constructed non-English prompts with a unique
correct single-token continuation. From layer to layer, transformers gradually
map an input embedding of the final prompt token to an output embedding from
which next-token probabilities are computed. Tracking intermediate embeddings
through their high-dimensional space reveals three distinct phases, whereby
intermediate embeddings (1) start far away from output token embeddings; (2)
already allow for decoding a semantically correct next token in the middle
layers, but give higher probability to its version in English than in the input
language; (3) finally move into an input-language-specific region of the
embedding space. We cast these results into a conceptual model where the three
phases operate in "input space", "concept space", and "output space",
respectively. Crucially, our evidence suggests that the abstract "concept
space" lies closer to English than to other languages, which may have important
consequences regarding the biases held by multilingual language models.

我们研究多语言语言模型在不平衡的以英语为主导的语料库上是否使用英语作为内部枢纽语言，以关键的重要性问题来理解语言模型的功能和语言偏见的起源。通过跟踪高维空间中的中间嵌入，我们发现了三个不同的阶段，这些阶段分别在 “输入空间”、“概念空间” 和 “输出空间” 中操作。关键的证据表明，抽象的 “概念空间” 更接近于英语而不是其他语言，这可能对多语言语言模型所持的偏见有重要影响。

多语言转换器的潜在语言：拉马在英文中起作用吗？

Do Llamas Work in English? On the Latent Language of Multilingual  Transformers

Human conversations naturally evolve around related concepts and scatter to
multi-hop concepts. This paper presents a new conversation generation model,
ConceptFlow, which leverages commonsense knowledge graphs to explicitly model
conversation flows. By grounding conversations to the concept space,
ConceptFlow represents the potential conversation flow as traverses in the
concept space along commonsense relations. The traverse is guided by graph
attentions in the concept graph, moving towards more meaningful directions in
the concept space, in order to generate more semantic and informative
responses. Experiments on Reddit conversations demonstrate ConceptFlow's
effectiveness over previous knowledge-aware conversation models and GPT-2 based
models while using 70% fewer parameters, confirming the advantage of explicit
modeling conversation structures. All source codes of this work are available
at this https URL

本篇论文提出一种新的对话生成模型，ConceptFlow，其利用常识知识图谱来显式地模拟对话流程，将对话基于概念空间来确定可能的转移方向，并通过图堆叠网络进行对话潜在流程的指导，以生成更有意义、更有语义信息的回应，证明了其在 Reddit 对话实验中的有效性，而且使用的参数数量比其他知识感知对话模型和基于 GPT-2 的模型少 70％。