Knowledge tagging for questions plays a crucial role in contemporary
intelligent educational applications, including learning progress diagnosis,
practice question recommendations, and course content organization.
Traditionally, these annotations are always conducted by pedagogical experts,
as the task requires not only a strong semantic understanding of both question
stems and knowledge definitions but also deep insights into connecting
question-solving logic with corresponding knowledge concepts. With the recent
emergence of advanced text encoding algorithms, such as pre-trained language
models, many researchers have developed automatic knowledge tagging systems
based on calculating the semantic similarity between the knowledge and question
embeddings. In this paper, we explore automating the task using Large Language
Models (LLMs), in response to the inability of prior encoding-based methods to
deal with the hard cases which involve strong domain knowledge and complicated
concept definitions. By showing the strong performance of zero- and few-shot
results over math questions knowledge tagging tasks, we demonstrate LLMs' great
potential in conquering the challenges faced by prior methods. Furthermore, by
proposing a reinforcement learning-based demonstration retriever, we
successfully exploit the great potential of different-sized LLMs in achieving
better performance results while keeping the in-context demonstration usage
efficiency high.

知识标记对于当代智能教育应用具有关键作用，本文探索采用大型语言模型（LLMs）自动化该任务，展示了 LLMs 在数学问题知识标记任务上零次和少次检测结果的强大性能，并通过提出基于强化学习的示范检索器，成功利用不同规模的 LLMs 实现更优异的性能结果同时保持上下文演示使用的高效性。

基于灵活演示检索器的数学问题知识标记系统

Knowledge Tagging System on Math Questions via LLMs with Flexible  Demonstration Retriever

Knowledge concept tagging for questions plays a crucial role in contemporary
intelligent educational applications, including learning progress diagnosis,
practice question recommendations, and course content organization.
Traditionally, these annotations have been conducted manually with help from
pedagogical experts, as the task requires not only a strong semantic
understanding of both question stems and knowledge definitions but also deep
insights into connecting question-solving logic with corresponding knowledge
concepts. In this paper, we explore automating the tagging task using Large
Language Models (LLMs), in response to the inability of prior manual methods to
meet the rapidly growing demand for concept tagging in questions posed by
advanced educational applications. Moreover, the zero/few-shot learning
capability of LLMs makes them well-suited for application in educational
scenarios, which often face challenges in collecting large-scale,
expertise-annotated datasets. By conducting extensive experiments with a
variety of representative LLMs, we demonstrate that LLMs are a promising tool
for concept tagging in math questions. Furthermore, through case studies
examining the results from different LLMs, we draw some empirical conclusions
about the key factors for success in applying LLMs to the automatic concept
tagging task.

基于大语言模型的知识概念标记在数学问题的自动标记任务中显示出潜力，由于其零 / 少样本学习能力，适用于教育场景中收集大规模、专业标记数据集所面临的挑战。