We present Sequence Salience, a visual tool for interactive prompt debugging
with input salience methods. Sequence Salience builds on widely used salience
methods for text classification and single-token prediction, and extends this
to a system tailored for debugging complex LLM prompts. Our system is
well-suited for long texts, and expands on previous work by 1) providing
controllable aggregation of token-level salience to the word, sentence, or
paragraph level, making salience over long inputs tractable; and 2) supporting
rapid iteration where practitioners can act on salience results, refine
prompts, and run salience on the new output. We include case studies showing
how Sequence Salience can help practitioners work with several complex
prompting strategies, including few-shot, chain-of-thought, and constitutional
principles. Sequence Salience is built on the Learning Interpretability Tool,
an open-source platform for ML model visualizations, and code, notebooks, and
tutorials are available at this http URL

我们提出了 Sequence Salience，一种用于交互式提示调试的视觉工具，结合了输入显著性方法，可帮助调试复杂的 LLM 提示策略。

交互式序列显著性调试

Interactive Prompt Debugging with Sequence Salience

Feature attribution a.k.a. input salience methods which assign an importance
score to a feature are abundant but may produce surprisingly different results
for the same model on the same input. While differences are expected if
disparate definitions of importance are assumed, most methods claim to provide
faithful attributions and point at the features most relevant for a model's
prediction. Existing work on faithfulness evaluation is not conclusive and does
not provide a clear answer as to how different methods are to be compared.
Focusing on text classification and the model debugging scenario, our main
contribution is a protocol for faithfulness evaluation that makes use of
partially synthetic data to obtain ground truth for feature importance ranking.
Following the protocol, we do an in-depth analysis of four standard salience
method classes on a range of datasets and shortcuts for BERT and LSTM models
and demonstrate that some of the most popular method configurations provide
poor results even for simplest shortcuts. We recommend following the protocol
for each new task and model combination to find the best method for identifying
shortcuts.

本文提出了一种研究模型特征重要性的方法，并对四种标准输入明显性方法在文本分类和模型调试场景下的效果进行了深入分析，并建议针对每种新任务 - 模型组合采用该方法以找到找到最佳的快捷方式识别方法。