Prompts play a crucial role in guiding the responses of Large Language Models
(LLMs). However, the intricate role of individual tokens in prompts, known as
input saliency, in shaping the responses remains largely underexplored.
Existing saliency methods either misalign with LLM generation objectives or
rely heavily on linearity assumptions, leading to potential inaccuracies. To
address this, we propose Token Distribution Dynamics (TDD), a
\textcolor{black}{simple yet effective} approach to unveil and manipulate the
role of prompts in generating LLM outputs. TDD leverages the robust
interpreting capabilities of the language model head (LM head) to assess input
saliency. It projects input tokens into the embedding space and then estimates
their significance based on distribution dynamics over the vocabulary. We
introduce three TDD variants: forward, backward, and bidirectional, each
offering unique insights into token relevance. Extensive experiments reveal
that the TDD surpasses state-of-the-art baselines with a big margin in
elucidating the causal relationships between prompts and LLM outputs. Beyond
mere interpretation, we apply TDD to two prompt manipulation tasks for
controlled text generation: zero-shot toxic language suppression and sentiment
steering. Empirical results underscore TDD's proficiency in identifying both
toxic and sentimental cues in prompts, subsequently mitigating toxicity or
modulating sentiment in the generated content.

通过使用 Token Distribution Dynamics 方法，我们揭示了 Prompt 输入对于生成大型语言模型 (如 GPT) 输出的重要作用，该方法显著优于现有的基线方法，并且在控制文本生成方面具有潜力应用于抑制有害语言和调节情感。