Understanding how semantic meaning is encoded in the representation spaces of large language models is a fundamental problem in interpretability. In this paper, we study the two foundational questions in this area. First, how are categorical concepts, such as {'mammal', 'bird', 'reptile', 'fish'}, represented? Second, how are hierarchical relations between concepts encoded? For example, how is the fact that 'dog' is a kind of 'mammal' encoded? We show how to extend the linear representation hypothesis to answer these questions. We find a remarkably simple structure: simple categorical concepts are represented as simplices, hierarchically related concepts are orthogonal in a sense we make precise, and (in consequence) complex concepts are represented as polytopes constructed from direct sums of simplices, reflecting the hierarchical structure. We validate these theoretical results on the Gemma large language model, estimating representations for 957 hierarchically related concepts using data from WordNet.

大型语言模型的表示空间中如何编码语义含义是可解释性中的一个根本问题。本文研究了这一领域的两个基本问题：第一，如何表示类别概念，如“哺乳动物”、“鸟类”、“爬行动物”、“鱼类”等；第二，如何编码概念之间的层级关系，例如“狗”是“哺乳动物”的一种。我们通过扩展线性表示假设来回答这些问题，并发现了一个非常简单的结构：简单的类别概念被表示为单纯形，具有层级关系的概念在某种意义上是正交的，并且（作为结果）复杂的概念被表示为由单纯形的直和构成的多面体，反映了其层级结构。我们使用来自WordNet的数据验证了这些理论结果，在Gemma大型语言模型上估计了957个具有层级关系的概念的表示。

大型语言模型中分类和层级概念的几何结构