A growing body of work makes use of probing to investigate the working of
neural models, often considered black boxes. Recently, an ongoing debate
emerged surrounding the limitations of the probing paradigm. In this work, we
point out the inability to infer behavioral conclusions from probing results
and offer an alternative method that focuses on how the information is being
used, rather than on what information is encoded. Our method, Amnesic Probing,
follows the intuition that the utility of a property for a given task can be
assessed by measuring the influence of a causal intervention that removes it
from the representation. Equipped with this new analysis tool, we can ask
questions that were not possible before, e.g. is part-of-speech information
important for word prediction? We perform a series of analyses on BERT to
answer these types of questions. Our findings demonstrate that conventional
probing performance is not correlated to task importance, and we call for
increased scrutiny of claims that draw behavioral or causal conclusions from
probing results.

本研究提出了 Amnesic Probing 方法，用以替代传统的伪推式分析方法，并通过对 BERT 进行一系列分析，发现伪推性能并不与任务重要性相关，并呼吁加强对从伪推结果中得出行为或因果结论的主张的审查。