Despite the popularity of Shapley Values in explaining neural text
classification models, computing them is prohibitive for large pretrained
models due to a large number of model evaluations. In practice, Shapley Values
are often estimated with a small number of stochastic model evaluations.
However, we show that the estimated Shapley Values are sensitive to random seed
choices -- the top-ranked features often have little overlap across different
seeds, especially on examples with longer input texts. This can only be
mitigated by aggregating thousands of model evaluations, which on the other
hand, induces substantial computational overheads. To mitigate the trade-off
between stability and efficiency, we develop an amortized model that directly
predicts each input feature's Shapley Value without additional model
evaluations. It is trained on a set of examples whose Shapley Values are
estimated from a large number of model evaluations to ensure stability.
Experimental results on two text classification datasets demonstrate that our
amortized model estimates Shapley Values accurately with up to 60 times speedup
compared to traditional methods. Furthermore, the estimated values are stable
as the inference is deterministic. We release our code at
this https URL

在神经网络文本分类模型中解释 Shapley Values 的流行度越来越高，但由于模型评估数量过多，计算它们对于大型预训练模型来说是禁止的。为了权衡稳定性和效率，我们开发了一种分散模型，直接预测每个输入特征的 Shapley Value，实验结果表明，与传统方法相比，我们的分散模型以高达 60 倍的速度准确地估计 Shapley 值。

通过分摊法实现文本分类的有效 Shapley 值估计

Efficient Shapley Values Estimation by Amortization for Text  Classification

Large pre-trained transformer-based language models have achieved impressive
results on a wide range of NLP tasks. In the past few years, Knowledge
Distillation(KD) has become a popular paradigm to compress a computationally
expensive model to a resource-efficient lightweight model. However, most KD
algorithms, especially in NLP, rely on the accessibility of the original
training dataset, which may be unavailable due to privacy issues. To tackle
this problem, we propose a novel two-stage data-free distillation method, named
Adversarial self-Supervised Data-Free Distillation (AS-DFD), which is designed
for compressing large-scale transformer-based models (e.g., BERT). To avoid
text generation in discrete space, we introduce a Plug & Play Embedding
Guessing method to craft pseudo embeddings from the teacher's hidden knowledge.
Meanwhile, with a self-supervised module to quantify the student's ability, we
adapt the difficulty of pseudo embeddings in an adversarial training manner. To
the best of our knowledge, our framework is the first data-free distillation
framework designed for NLP tasks. We verify the effectiveness of our method on
several text classification datasets.

提出一种名为 AS-DFD 的新的两阶段无数据蒸馏方法，用于压缩大型基于 Transformer 的模型（例如 BERT），并且是第一个面向 NLP 任务设计的无数据蒸馏框架，在 Text Classification 数据集上验证了其有效性。