Recent work exploring the capabilities of pre-trained large language models
(LLMs) has demonstrated their ability to act as general pattern machines by
completing complex token sequences representing a wide array of tasks,
including time-series prediction and robot control. This paper investigates
whether the pattern recognition and sequence modeling capabilities of LLMs can
be extended to the domain of knowledge tracing, a critical component in the
development of intelligent tutoring systems (ITSs) that tailor educational
experiences by predicting learner performance over time. In an empirical
evaluation across multiple real-world datasets, we compare two approaches to
using LLMs for this task, zero-shot prompting and model fine-tuning, with
existing, non-LLM approaches to knowledge tracing. While LLM-based approaches
do not achieve state-of-the-art performance, fine-tuned LLMs surpass the
performance of naive baseline models and perform on par with standard Bayesian
Knowledge Tracing approaches across multiple metrics. These findings suggest
that the pattern recognition capabilities of LLMs can be used to model complex
learning trajectories, opening a novel avenue for applying LLMs to educational
contexts. The paper concludes with a discussion of the implications of these
findings for future research, suggesting that further refinements and a deeper
understanding of LLMs' predictive mechanisms could lead to enhanced performance
in knowledge tracing tasks.

本文探讨了预训练大型语言模型（LLMs）的能力，以及它们在知识跟踪、智能辅导系统领域的应用，研究了两种使用 LLMs 进行知识跟踪的方法，并对其在实际数据集上的性能进行了评估，结果表明 LLMs 能够模拟复杂的学习轨迹，为将 LLMs 应用于教育环境开辟了新的途径。

基于大型语言模型的学习者表现建模

Towards Modeling Learner Performance with Large Language Models

Text-rich VQA, namely Visual Question Answering based on text recognition in
the images, is a cross-modal task that requires both image comprehension and
text recognition. In this work, we focus on investigating the advantages and
bottlenecks of LLM-based approaches in addressing this problem. To address the
above concern, we separate the vision and language modules, where we leverage
external OCR models to recognize texts in the image and Large Language Models
(LLMs) to answer the question given texts. The whole framework is training-free
benefiting from the in-context ability of LLMs. This pipeline achieved superior
performance compared to the majority of existing Multimodal Large Language
Models (MLLM) on four text-rich VQA datasets. Besides, based on the ablation
study, we find that LLM brings stronger comprehension ability and may introduce
helpful knowledge for the VQA problem. The bottleneck for LLM to address
text-rich VQA problems may primarily lie in visual part. We also combine the
OCR module with MLLMs and pleasantly find that the combination of OCR module
with MLLM also works. It's worth noting that not all MLLMs can comprehend the
OCR information, which provides insights into how to train an MLLM that
preserves the abilities of LLM.

基于文本识别的图像视觉问答是一个跨模态任务，需要图像理解和文本识别。本文研究了基于 LLM 方法在解决此问题时的优势和瓶颈，并通过整合 OCR 模块和 MLLM 发现多数 MLLM 可以理解 OCR 信息，为训练保留 LLM 能力提供了启示。