Free-text rationales justify model decisions in natural language and thus become likable and accessible among approaches to explanation across many tasks. However, their effectiveness can be hindered by misinterpretation and hallucination. As a perturbation test, we investigate how large language models (LLMs) perform the task of natural language explanation (NLE) under the effects of readability level control, i.e., being prompted for a rationale targeting a specific expertise level, such as sixth grade or college. We find that explanations are adaptable to such instruction, but the requested readability is often misaligned with the measured text complexity according to traditional readability metrics. Furthermore, the quality assessment shows that LLMs' ratings of rationales across text complexity exhibit a similar pattern of preference as observed in natural language generation (NLG). Finally, our human evaluation suggests a generally satisfactory impression on rationales at all readability levels, with high-school-level readability being most commonly perceived and favored.

大型语言模型在不同可读性级别下进行自然语言解释任务，发现解释可以适应不同的指导，但请求的可读性经常与传统可读性评估指标不一致。该研究还发现，评估大型语言模型生成的解释时，其对于文本复杂度的评估呈现出与自然语言生成中观察到的类似偏好。最后，人类评估表明在不同可读性级别下的解释总体印象令人满意，其中高中级别的可读性最常见且受欢迎。

在可读性水平控制下的自由文本论据生成