Feature attribution methods (FAs), such as gradients and attention, are
widely employed approaches to derive the importance of all input features to
the model predictions. Existing work in natural language processing has mostly
focused on developing and testing FAs for encoder-only language models (LMs) in
classification tasks. However, it is unknown if it is faithful to use these FAs
for decoder-only models on text generation, due to the inherent differences
between model architectures and task settings respectively. Moreover, previous
work has demonstrated that there is no `one-wins-all' FA across models and
tasks. This makes the selection of a FA computationally expensive for large LMs
since input importance derivation often requires multiple forward and backward
passes including gradient computations that might be prohibitive even with
access to large compute. To address these issues, we present a model-agnostic
FA for generative LMs called Recursive Attribution Generator (ReAGent). Our
method updates the token importance distribution in a recursive manner. For
each update, we compute the difference in the probability distribution over the
vocabulary for predicting the next token between using the original input and
using a modified version where a part of the input is replaced with RoBERTa
predictions. Our intuition is that replacing an important token in the context
should have resulted in a larger change in the model's confidence in predicting
the token than replacing an unimportant token. Our method can be universally
applied to any generative LM without accessing internal model weights or
additional training and fine-tuning, as most other FAs require. We extensively
compare the faithfulness of ReAGent with seven popular FAs across six
decoder-only LMs of various sizes. The results show that our method
consistently provides more faithful token importance distributions.

我们提出了一种递归归因生成器（ReAGent）方法，用于生成语言模型的模型无关特征归因，通过计算原始输入和替换部分输入进行比较，确保我们的方法在各种解码器模型中提供更加准确的关键词重要性分布。

ReAGent：面向生成语言模型的无模型特征归因方法

ReAGent: Towards A Model-agnostic Feature Attribution Method for  Generative Language Models

Pre-trained language models (LMs) have been shown to memorize a substantial
amount of knowledge from the pre-training corpora; however, they are still
limited in recalling factually correct knowledge given a certain context.
Hence, they tend to suffer from counterfactual or hallucinatory generation when
used in knowledge-intensive natural language generation (NLG) tasks. Recent
remedies to this problem focus on modifying either the pre-training or task
fine-tuning objectives to incorporate knowledge, which normally require
additional costly training or architecture modification of LMs for practical
applications. We present Knowledge Infused Decoding (KID) -- a novel decoding
algorithm for generative LMs, which dynamically infuses external knowledge into
each step of the LM decoding. Specifically, we maintain a local knowledge
memory based on the current context, interacting with a dynamically created
external knowledge trie, and continuously update the local memory as a
knowledge-aware constraint to guide decoding via reinforcement learning. On six
diverse knowledge-intensive NLG tasks, task-agnostic LMs (e.g., GPT-2 and BART)
armed with KID outperform many task-optimized state-of-the-art models, and show
particularly strong performance in few-shot scenarios over seven related
knowledge-infusion techniques. Human evaluation confirms KID's ability to
generate more relevant and factual language for the input context when compared
with multiple baselines. Finally, KID also alleviates exposure bias and
provides stable generation quality when generating longer sequences. Code for
KID is available at this https URL.

通过一种名为 KID 的基于强化学习的算法，将外部知识动态地注入到生成语言模型的解码过程中，以解决语境相关性和事实准确性的问题，进而在多种知识密集型自然语言生成任务中体现出比许多优化后的模型更好的性能表现。

知识注入解码

Knowledge Infused Decoding

Large pre-trained neural language models (LM) have very powerful text
generation capabilities. However, in practice, they are hard to control for
creative purposes. We describe a Plug-and-Play controllable language generation
framework, Plug-and-Blend, that allows a human user to input multiple control
codes (topics). In the context of automated story generation, this allows a
human user loose or fine-grained control of the topics and transitions between
them that will appear in the generated story, and can even allow for
overlapping, blended topics. Automated evaluations show our framework, working
with different generative LMs, controls the generation towards given
continuous-weighted control codes while keeping the generated sentences fluent,
demonstrating strong blending capability. A human participant evaluation shows
that the generated stories are observably transitioning between two topics.

通过 Plug-and-Blend 框架在人工输入多个控制代码的情况下，通过使用不同的生成语言模型进行自动故事生成并不断优化和生成流畅的故事，实现了主题的宽松和细粒度控制，并实现了叠加和混合主题。