Modern generative models demonstrate impressive capabilities, likely stemming
from an ability to identify and manipulate abstract concepts underlying their
training data. However, fundamental questions remain: what determines the
concepts a model learns, the order in which it learns them, and its ability to
manipulate those concepts? To address these questions, we propose analyzing a
model's learning dynamics via a framework we call the concept space, where each
axis represents an independent concept underlying the data generating process.
By characterizing learning dynamics in this space, we identify how the speed at
which a concept is learned, and hence the order of concept learning, is
controlled by properties of the data we term concept signal. Further, we
observe moments of sudden turns in the direction of a model's learning dynamics
in concept space. Surprisingly, these points precisely correspond to the
emergence of hidden capabilities, i.e., where latent interventions show the
model possesses the capability to manipulate a concept, but these capabilities
cannot yet be elicited via naive input prompting. While our results focus on
synthetically defined toy datasets, we hypothesize a general claim on emergence
of hidden capabilities may hold: generative models possess latent capabilities
that emerge suddenly and consistently during training, though a model might not
exhibit these capabilities under naive input prompting.

通过新的概念空间框架分析模型的学习动态，在合成定义的玩具数据集上发现生成模型具备突然且一致地在训练过程中出现的潜在能力，虽然模型可能在普通输入提示下无法展示这些能力。