Effective collaboration requires groups to strategically regulate themselves
to overcome challenges. Research has shown that groups may fail to regulate due
to differences in members' perceptions of challenges which may benefit from
external support. In this study, we investigated the potential of leveraging
three distinct natural language processing models: an expert knowledge
rule-based model, a supervised machine learning (ML) model and a Large Language
model (LLM), in challenge detection and challenge dimension identification
(cognitive, metacognitive, emotional and technical/other challenges) from
student discourse, was investigated. The results show that the supervised ML
and the LLM approaches performed considerably well in both tasks, in contrast
to the rule-based approach, whose efficacy heavily relies on the engineered
features by experts. The paper provides an extensive discussion of the three
approaches' performance for automated detection and support of students'
challenge moments in collaborative learning activities. It argues that,
although LLMs provide many advantages, they are unlikely to be the panacea to
issues of the detection and feedback provision of socially shared regulation of
learning due to their lack of reliability, as well as issues of validity
evaluation, privacy and confabulation. We conclude the paper with a discussion
on additional considerations, including model transparency to explore feasible
and meaningful analytical feedback for students and educators using LLMs.

本研究调查了利用专家知识规则模型、监督式机器学习模型和大型语言模型（LLM）从学生讨论中检测挑战和识别挑战维度（认知、元认知、情感和技术 / 其他挑战）的潜力。结果显示，监督式机器学习和 LLM 在两个任务中表现出色，而基于规则的方法则主要依赖专家设计的特征。该论文对自动检测和支持学生在协作学习活动中遇到的挑战时刻的三种方法的表现进行了广泛讨论。论文认为，尽管 LLMs 具有许多优势，但由于其缺乏可靠性以及在有效性评估、隐私和杂乱性方面存在问题，它们不太可能成为解决学习的社交共享调控的检测和反馈问题的万灵药。我们通过讨论额外的考虑因素来结束该论文，包括模型的透明度，以探索使用 LLMs 为学生和教育工作者提供可行且有意义的分析反馈。