Probing experiments investigate the extent to which neural representations
make properties -- like part-of-speech -- predictable. One suggests that a
representation encodes a property if probing that representation produces
higher accuracy than probing a baseline representation like non-contextual word
embeddings. Instead of using baselines as a point of comparison, we're
interested in measuring information that is contained in the representation but
not in the baseline. For example, current methods can detect when a
representation is more useful than the word identity (a baseline) for
predicting part-of-speech; however, they cannot detect when the representation
is predictive of just the aspects of part-of-speech not explainable by the word
identity. In this work, we extend a theory of usable information called
$\mathcal{V}$-information and propose conditional probing, which explicitly
conditions on the information in the baseline. In a case study, we find that
after conditioning on non-contextual word embeddings, properties like
part-of-speech are accessible at deeper layers of a network than previously
thought.

研究神经表示在探测实验中的有效性并提出一种条件探测的理论，通过该理论研究发现在条件化非上下文单词嵌入后，部分性质如词性是可以在深层网络中被探测的。