Assessing student answers and providing valuable feedback is crucial for
effective learning, but it can be a time-consuming task. Traditional methods of
automating student answer assessment through text classification often suffer
from issues such as lack of trustworthiness, transparency, and the ability to
provide a rationale for the automated assessment process. These limitations
hinder their usefulness in practice. In this paper, we explore using ChatGPT, a
cutting-edge large language model, for the concurrent tasks of student answer
scoring and rationale generation under both the zero-shot and few-shot
settings. We introduce a critic module which automatically filters incorrect
outputs from ChatGPT and utilizes the remaining ChtaGPT outputs as noisy
labelled data to fine-tune a smaller language model, enabling it to perform
student answer scoring and rationale generation. Moreover, by drawing multiple
samples from ChatGPT outputs, we are able to compute predictive confidence
scores, which in turn can be used to identify corrupted data and human label
errors in the training set. Our experimental results demonstrate that despite
being a few orders of magnitude smaller than ChatGPT, the fine-tuned language
model achieves better performance in student answer scoring. Furthermore, it
generates more detailed and comprehensible assessments than traditional text
classification methods. Our approach provides a viable solution to achieve
explainable automated assessment in education.

本文探讨使用 ChatGPT 模型进行学生答案自动评分和理由生成的方法，并通过引入批评模块，过滤 ChatGPT 的不正确输出并细调较小的语言模型，在学生答案评分和理由生成方面实现了更好的性能，并且能够提供更详细和易理解的评估结果，从而提供了一个可行的解决方案，实现了可解释的自动评估教育。