Large Language Models (LLMs) have recently made impressive strides in natural
language understanding tasks. Despite their remarkable performance,
understanding their decision-making process remains a big challenge. In this
paper, we look into bringing some transparency to this process by introducing a
new explanation dataset for question answering (QA) tasks that integrates
knowledge graphs (KGs) in a novel way. Our dataset includes 12,102
question-answer-explanation (QAE) triples. Each explanation in the dataset
links the LLM's reasoning to entities and relations in the KGs. The explanation
component includes a why-choose explanation, a why-not-choose explanation, and
a set of reason-elements that underlie the LLM's decision. We leverage KGs and
graph attention networks (GAT) to find the reason-elements and transform them
into why-choose and why-not-choose explanations that are comprehensible to
humans. Through quantitative and qualitative evaluations, we demonstrate the
potential of our dataset to improve the in-context learning of LLMs, and
enhance their interpretability and explainability. Our work contributes to the
field of explainable AI by enabling a deeper understanding of the LLMs
decision-making process to make them more transparent and thereby, potentially
more reliable, to researchers and practitioners alike. Our dataset is available
at: this https URL

大语言模型（LLMs）在自然语言理解任务中取得了显著进展，但是理解它们的决策过程仍然是一个巨大的挑战。本文通过引入一种新的解释数据集，将知识图谱（KGs）与问答（QA）任务以一种新颖的方式集成，为这一过程带来了一定的透明度。通过定量和定性评估，我们展示了我们的数据集提高了 LLMs 的上下文学习能力，并增强了它们的可解释性。我们的工作通过深入理解 LLMs 的决策过程，使其更加透明，从而对研究人员和从业人员更可靠，为可解释 AI 领域做出贡献。

XplainLLM：理解 LLM 决策的 QA 解释数据集

XplainLLM: A QA Explanation Dataset for Understanding LLM  Decision-Making

Human label variation (Plank 2022), or annotation disagreement, exists in
many natural language processing (NLP) tasks. To be robust and trusted, NLP
models need to identify such variation and be able to explain it. To this end,
we created the first ecologically valid explanation dataset with diverse
reasoning, LiveNLI. LiveNLI contains annotators' highlights and free-text
explanations for the label(s) of their choice for 122 English Natural Language
Inference items, each with at least 10 annotations. We used its explanations
for chain-of-thought prompting, and found there is still room for improvement
in GPT-3's ability to predict label distribution with in-context learning.

该论文创建了一个包含注解高亮和自由文本解释的生态有效的自然语言推断数据集 --LiveNLI。使用该数据集对思维链进行提示，发现目前 GPT-3 在预测标签分布方面仍有改进的空间