Using large language models (LLMs) for educational applications like
dialogue-based teaching is a hot topic. Effective teaching, however, requires
teachers to adapt the difficulty of content and explanations to the education
level of their students. Even the best LLMs today struggle to do this well. If
we want to improve LLMs on this adaptation task, we need to be able to measure
adaptation success reliably. However, current Static metrics for text
difficulty, like the Flesch-Kincaid Reading Ease score, are known to be crude
and brittle. We, therefore, introduce and evaluate a new set of Prompt-based
metrics for text difficulty. Based on a user study, we create Prompt-based
metrics as inputs for LLMs. They leverage LLM's general language understanding
capabilities to capture more abstract and complex features than Static metrics.
Regression experiments show that adding our Prompt-based metrics significantly
improves text difficulty classification over Static metrics alone. Our results
demonstrate the promise of using LLMs to evaluate text adaptation to different
education levels.

使用大型语言模型（LLMs）对教育应用，如基于对话的教学，进行研究是一个热门话题。本研究引入并评估了基于提示的文本难度度量，该度量方法相比静态度量具有更好的性能，并展示了利用 LLMs 评估文本对不同教育水平的适应的潜力。

超越 Flesch-Kincaid：基于提示的度量改善教育文本的难度分类

Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty  Classification of Educational Texts

Large language models (LLMs) offer a range of new possibilities, including
adapting the text to different audiences and their reading needs. But how well
do they adapt? We evaluate the readability of answers generated by four
state-of-the-art LLMs (commercial and open-source) to science questions when
prompted to target different age groups and education levels. To assess the
adaptability of LLMs to diverse audiences, we compare the readability scores of
the generated responses against the recommended comprehension level of each age
and education group. We find large variations in the readability of the answers
by different LLMs. Our results suggest LLM answers need to be better adapted to
the intended audience demographics to be more comprehensible. They underline
the importance of enhancing the adaptability of LLMs in education settings to
cater to diverse age and education levels. Overall, current LLMs have set
readability ranges and do not adapt well to different audiences, even when
prompted. That limits their potential for educational purposes.

对四种最先进的大型语言模型（商业和开源）回答科学问题并针对不同年龄组和教育水平的读者群体进行可读性评估，发现它们的可读性存在较大差异。研究结果表明，目前的大型语言模型在适应不同受众时的可读性有待提高，强调了在教育场景中增强大型语言模型的适应性以迎合不同年龄和教育水平的重要性，当前大型语言模型的可读性范围受限，无法很好地适应不同受众，即使在提示的前提下，这限制了它们在教育用途上的潜力。