This paper presents a novel exploration into the regressive side effects of
training Large Language Models (LLMs) to mimic student misconceptions for
personalized education. We highlight the problem that as LLMs are trained to
more accurately mimic student misconceptions, there is a compromise in the
factual integrity and reasoning ability of the models. Our work involved
training an LLM on a student-tutor dialogue dataset to predict student
responses. The results demonstrated a decrease in the model's performance
across multiple benchmark datasets, including the ARC reasoning challenge and
TruthfulQA, which evaluates the truthfulness of model's generated responses.
Furthermore, the HaluEval Dial dataset, used for hallucination detection, and
MemoTrap, a memory-based task dataset, also reported a decline in the model
accuracy. To combat these side effects, we introduced a "hallucination token"
technique. This token, appended at the beginning of each student response
during training, instructs the model to switch between mimicking student
misconceptions and providing factually accurate responses. Despite the
significant improvement across all datasets, the technique does not completely
restore the LLM's baseline performance, indicating the need for further
research in this area. This paper contributes to the ongoing discussion on the
use of LLMs for student modeling, emphasizing the need for a balance between
personalized education and factual accuracy.

对于使用大型语言模型（LLM）模仿学生误解以进行个性化教育，本研究探讨了其退化副作用。我们发现，随着 LLM 被训练得更准确地模仿学生误解，模型的事实真实性和推理能力会受到妥协。为了解决这些副作用，我们引入了一种 “幻觉标记” 技术，并发现在多个基准数据集上效果显著提升，但仍需要进一步研究保持 LLM 在个性化教育和事实准确性之间的平衡。