In this study, we developed an automated short answer grading (ASAG) model
that provided both analytic scores and final holistic scores. Short answer
items typically consist of multiple sub-questions, and providing an analytic
score and the text span relevant to each sub-question can increase the
interpretability of the automated scores. Furthermore, they can be used to
generate actionable feedback for students. Despite these advantages, most
studies have focused on predicting only holistic scores due to the difficulty
in constructing dataset with manual annotations. To address this difficulty, we
used large language model (LLM)-based one-shot prompting and a text similarity
scoring model with domain adaptation using small manually annotated dataset.
The accuracy and quadratic weighted kappa of our model were 0.67 and 0.71 on a
subset of the publicly available ASAG dataset. The model achieved a substantial
improvement over the majority baseline.

本文研究了一种自动短答案评分（ASAG）模型，提供了分析性得分和最终的整体得分。使用基于大型语言模型（LLM）的一次提示和文本相似性评分模型，通过小型手动注释数据集进行领域适应性，获得了较高的准确性和加权 kappa 值，从而在公开 ASAG 数据集的子集上取得了重大进展。