The advent of black-box deep neural network classification models has sparked
the need to explain their decisions. However, in the case of generative AI such
as large language models (LLMs), there is no class prediction to explain.
Rather, one can ask why an LLM output a particular response to a given prompt.
In this paper, we answer this question by proposing, to the best of our
knowledge, the first contrastive explanation methods requiring simply
black-box/query access. Our explanations suggest that an LLM outputs a reply to
a given prompt because if the prompt was slightly modified, the LLM would have
given a different response that is either less preferable or contradicts the
original response. The key insight is that contrastive explanations simply
require a distance function that has meaning to the user and not necessarily a
real valued representation of a specific response (viz. class label). We offer
two algorithms for finding contrastive explanations: i) A myopic algorithm,
which although effective in creating contrasts, requires many model calls and
ii) A budgeted algorithm, our main algorithmic contribution, which
intelligently creates contrasts adhering to a query budget, necessary for
longer contexts. We show the efficacy of these methods on diverse natural
language tasks such as open-text generation, automated red teaming, and
explaining conversational degradation.

通过黑盒 / 查询访问，我们提出了两种对比解释方法：一种是效率较高但需要多次模型调用的近视算法，另一种是主要算法贡献的预算算法。我们展示了这些方法在不同自然语言任务中的有效性，例如开放文本生成、自动红队战术和解释对话衰退。