Interventions targeting the representation space of language models (LMs)
have emerged as effective means to influence model behavior. These methods are
employed, for example, to eliminate or alter the encoding of demographic
information such as gender within the model's representations, creating a
counterfactual representation. However, since the intervention operates within
the representation space, understanding precisely which features it modifies
poses a challenge. We show that representation-space counterfactuals can be
converted into natural language counterfactuals. We demonstrate that this
approach enables us to analyze the linguistic alterations corresponding to a
given representation-space intervention and to interpret the features utilized
for encoding a specific concept. Moreover, the resulting counterfactuals can be
used to mitigate bias in classification.

通过操作语言模型的表示空间，我们可以将表示空间反事实转化为自然语言的反事实，从而分析与特定表示空间干预对应的语言变化，并解释用于编码特定概念的特征。此外，生成的反事实可用于减轻分类中的偏见。