With the growing popularity of deep-learning models, model understanding becomes more important. Much effort has been devoted to demystify deep neural networks for better interpretability. Some feature attribution methods have shown promising results in computer vision, especially the gradient-based methods where effectively smoothing the gradients with reference data is key to a robust and faithful result. However, direct application of these gradient-based methods to NLP tasks is not trivial due to the fact that the input consists of discrete tokens and the "reference" tokens are not explicitly defined. In this work, we propose Locally Aggregated Feature Attribution (LAFA), a novel gradient-based feature attribution method for NLP models. Instead of relying on obscure reference tokens, it smooths gradients by aggregating similar reference texts derived from language model embeddings. For evaluation purpose, we also design experiments on different NLP tasks including Entity Recognition and Sentiment Analysis on public datasets as well as key feature detection on a constructed Amazon catalogue dataset. The superior performance of the proposed method is demonstrated through experiments.

本文提出了用于自然语言处理模型的新型基于梯度的特征暗示方法（Locally Aggregated Feature Attribution，LAFA），通过汇集从语言模型嵌入推导出的类似参考文本来平滑梯度，以实现对模型的解释性。该方法在公共数据集上进行了实验，包括实体识别和情感分析以及在构建的Amazon目录数据集上对重要特征的检测。通过实验，证明了该方法的优越性能。

自然语言模型理解中的本地聚合特征归因