One-on-one tutoring is an effective instructional method for enhancing
learning, yet its efficacy hinges on tutor competencies. Novice math tutors
often prioritize content-specific guidance, neglecting aspects such as
social-emotional learning. Social-emotional learning promotes equity and
inclusion and nurturing relationships with students, which is crucial for
holistic student development. Assessing the competencies of tutors accurately
and efficiently can drive the development of tailored tutor training programs.
However, evaluating novice tutor ability during real-time tutoring remains
challenging as it typically requires experts-in-the-loop. To address this
challenge, this preliminary study aims to harness Generative Pre-trained
Transformers (GPT), such as GPT-3.5 and GPT-4 models, to automatically assess
tutors' ability of using social-emotional tutoring strategies. Moreover, this
study also reports on the financial dimensions and considerations of employing
these models in real-time and at scale for automated assessment. The current
study examined four prompting strategies: two basic Zero-shot prompt
strategies, Tree of Thought prompt, and Retrieval-Augmented Generator (RAG)
based prompt. The results indicate that the RAG prompt demonstrated more
accurate performance (assessed by the level of hallucination and correctness in
the generated assessment texts) and lower financial costs than the other
strategies evaluated. These findings inform the development of personalized
tutor training interventions to enhance the the educational effectiveness of
tutored learning.

通过使用 GPT 模型自动评估教师使用社交情感辅导策略的能力，研究发现 RAG 提示方式比其他评估策略具有更准确的表现和更低的财务成本，这为个性化教师培训干预的发展提供了指导。