Causal probing is an approach to interpreting foundation models, such as large language models, by training probes to recognize latent properties of interest from embeddings, intervening on probes to modify this representation, and analyzing the resulting changes in the model's behavior. While some recent works have cast doubt on the theoretical basis of several leading causal probing intervention methods, it has been unclear how to systematically and empirically evaluate their effectiveness in practice. To address this problem, we propose a general empirical analysis framework to evaluate the reliability of causal probing interventions, formally defining and quantifying two key causal probing desiderata: completeness (fully transforming the representation of the target property) and selectivity (minimally impacting other properties). Our formalism allows us to make the first direct comparisons between different families of causal probing methods (e.g., linear vs. nonlinear or counterfactual vs. nullifying interventions). We conduct extensive experiments across several leading methods, finding that (1) there is an inherent tradeoff between these criteria, and no method is able to consistently satisfy both at once; and (2) across the board, nullifying interventions are always far less complete than counterfactual interventions, indicating that nullifying methods may not be an effective approach to causal probing.

本研究解决了因果探测方法在理论基础上的有效性评估缺乏系统和实证分析的问题。提出了一种通用的实证分析框架，明确界定和量化了因果探测的两个关键要求：完整性和选择性。研究发现这两者之间存在固有的权衡，无论哪种方法都无法同时满足这两个标准，而无效干预的方法在完整性上始终远低于反事实干预，表明其在因果探测中的有效性受限。

评估因果探测方法的可靠性：权衡、限制与无效干预的困境