Recent causal probing literature reveals when language models and syntactic
probes use similar representations. Such techniques may yield "false negative"
causality results: models may use representations of syntax, but probes may
have learned to use redundant encodings of the same syntactic information. We
demonstrate that models do encode syntactic information redundantly and
introduce a new probe design that guides probes to consider all syntactic
information present in embeddings. Using these probes, we find evidence for the
use of syntax in models where prior methods did not, allowing us to boost model
performance by injecting syntactic information into representations.

语言模型中使用的语法信息可能存在多余编码，通过一种新的探针设计，可准确引导探针考虑嵌入中的所有语法信息，揭示语法在当前方法未能探测到的模型中的有效性，进而通过注入语法信息提高模型性能。

语法何时在神经语言模型性能中发挥作用？来自 Dropout 探针的证据

When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes

Measuring what linguistic information is encoded in neural models of language
has become popular in NLP. Researchers approach this enterprise by training
"probes" - supervised models designed to extract linguistic structure from
another model's output. One such probe is the structural probe (Hewitt and
Manning, 2019), designed to quantify the extent to which syntactic information
is encoded in contextualised word representations. The structural probe has a
novel design, unattested in the parsing literature, the precise benefit of
which is not immediately obvious. To explore whether syntactic probes would do
better to make use of existing techniques, we compare the structural probe to a
more traditional parser with an identical lightweight parameterisation. The
parser outperforms structural probe on UUAS in seven of nine analysed
languages, often by a substantial amount (e.g. by 11.1 points in English).
Under a second less common metric, however, there is the opposite trend - the
structural probe outperforms the parser. This begs the question: which metric
should we prefer?

本研究对比了一个新型结构探针和传统解析器，发现两种方法在不同语言上效果并存，提出了关于使用哪种技术的问题。