Model interpretability methods are often used to explain NLP model decisions
on tasks such as text classification, where the output space is relatively
small. However, when applied to language generation, where the output space
often consists of tens of thousands of tokens, these methods are unable to
provide informative explanations. Language models must consider various
features to predict a token, such as its part of speech, number, tense, or
semantics. Existing explanation methods conflate evidence for all these
features into a single explanation, which is less interpretable for human
understanding.
To disentangle the different decisions in language modeling, we focus on
explaining language models contrastively: we look for salient input tokens that
explain why the model predicted one token instead of another. We demonstrate
that contrastive explanations are quantifiably better than non-contrastive
explanations in verifying major grammatical phenomena, and that they
significantly improve contrastive model simulatability for human observers. We
also identify groups of contrastive decisions where the model uses similar
evidence, and we are able to characterize what input tokens models use during
various language generation decisions.

该研究关注对语言模型的对比解释，证明对比解释在验证大型语法现象和改善对相互模拟性上是可量化地更好的，并且可以表征模型在各种语言生成决策中使用的输入令牌。

利用对比解释解读语言模型

Interpreting Language Models with Contrastive Explanations

We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLiMP),
a challenge set for evaluating what language models (LMs) know about major
grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars, and aggregate human agreement with the labels is
96.4%. We use it to evaluate n-gram, LSTM, and Transformer (GPT-2 and
Transformer-XL) LMs. We find that state-of-the-art models identify
morphological contrasts reliably, but they struggle with semantic restrictions
on the distribution of quantifiers and negative polarity items and subtle
syntactic phenomena such as extraction islands.

BLiMP 是一套挑战集，用于评估语言模型对英语中主要语法现象的理解水平。研究表明，现有模型能够可靠地识别形态对比，但在限定词和否定极性项分布以及提取岛等微妙的语法现象上仍面临挑战。