As the importance of comprehensive evaluation in workshop courses increases,
there is a growing demand for efficient and fair assessment methods that reduce
the workload for faculty members. This paper presents an evaluation conducted
with Large Language Models (LLMs) using actual student essays in three
scenarios: 1) without providing guidance such as rubrics, 2) with pre-specified
rubrics, and 3) through pairwise comparison of essays. Quantitative analysis of
the results revealed a strong correlation between LLM and faculty member
assessments in the pairwise comparison scenario with pre-specified rubrics,
although concerns about the quality and stability of evaluations remained.
Therefore, we conducted a qualitative analysis of LLM assessment comments,
showing that: 1) LLMs can match the assessment capabilities of faculty members,
2) variations in LLM assessments should be interpreted as diversity rather than
confusion, and 3) assessments by humans and LLMs can differ and complement each
other. In conclusion, this paper suggests that LLMs should not be seen merely
as assistants to faculty members but as partners in evaluation committees and
outlines directions for further research.

本文通过三种场景下的评估：1）不提供指导，2）使用预先规定的评分标准，3）通过论文的两两对比，与实际学生论文一起使用大语言模型（LLM）进行评估，以降低教师的工作量。定量分析结果显示，使用预先规定的评分标准对 LLM 与教师评估之间存在强相关性，尽管存在有关评估质量和稳定性的担忧。因此，对 LLM 的评估意见进行了定性分析，结果表明：1）LLM 可以达到教师的评估能力，2）LLM 评估中的差异应解释为多样性而非混乱，3）人类和 LLM 的评估可以不同且相互补充。综上所述，本文建议将 LLM 视为教师评估委员会的合作伙伴，并为进一步研究提供了方向。