Large language models (LLMs) can be used to generate natural language
explanations (NLE) that are adapted to different users' situations. However,
there is yet to be a quantitative evaluation of the extent of such adaptation.
To bridge this gap, we collect a benchmarking dataset, Situation-Based
Explanation. This dataset contains 100 explanandums. Each explanandum is paired
with explanations targeted at three distinct audience types-such as educators,
students, and professionals-enabling us to assess how well the explanations
meet the specific informational needs and contexts of these diverse groups e.g.
students, teachers, and parents. For each "explanandum paired with an audience"
situation, we include a human-written explanation. These allow us to compute
scores that quantify how the LLMs adapt the explanations to the situations. On
an array of pretrained language models with varying sizes, we examine three
categories of prompting methods: rule-based prompting, meta-prompting, and
in-context learning prompting. We find that 1) language models can generate
prompts that result in explanations more precisely aligned with the target
situations, 2) explicitly modeling an "assistant" persona by prompting "You are
a helpful assistant..." is not a necessary prompt technique for situated NLE
tasks, and 3) the in-context learning prompts only can help LLMs learn the
demonstration template but can't improve their inference performance. SBE and
our analysis facilitate future research towards generating situated natural
language explanations.

利用大型语言模型，针对不同用户情境生成自然语言解释，并通过定量评估展示适应程度。我们收集了一个基准数据集，包括 100 个解释对象，每个对象都与针对三个不同受众类型（如教育工作者、学生和专业人员）的解释配对，使我们能够评估解释在特定信息需求和背景下让这些多样化群体（如学生、教师和家长）满意的程度。通过预训练语言模型，在不同大小的预训练语言模型上，我们考察了三类提示方法：基于规则的提示，元提示和上下文学习提示。我们发现：1）语言模型可以生成与目标情境更准确对齐的提示产生的解释；2）通过提示 “您是一个乐于助人的助手…” 来明确建模 “助手” 角色不是定位自然语言解释任务的必要技术；3）只有上下文学习提示能够帮助语言模型学习演示模板，但无法提高它们的推理性能。该基准数据集及我们的分析有助于未来生成情境化自然语言解释的研究。