Advances in automated essay scoring (AES) have traditionally relied on
labeled essays, requiring tremendous cost and expertise for their acquisition.
Recently, large language models (LLMs) have achieved great success in various
tasks, but their potential is less explored in AES. In this paper, we propose
Multi Trait Specialization (MTS), a zero-shot prompting framework to elicit
essay scoring capabilities in LLMs. Specifically, we leverage ChatGPT to
decompose writing proficiency into distinct traits and generate scoring
criteria for each trait. Then, an LLM is prompted to extract trait scores from
several conversational rounds, each round scoring one of the traits based on
the scoring criteria. Finally, we derive the overall score via trait averaging
and min-max scaling. Experimental results on two benchmark datasets demonstrate
that MTS consistently outperforms straightforward prompting (Vanilla) in
average QWK across all LLMs and datasets, with maximum gains of 0.437 on
TOEFL11 and 0.355 on ASAP. Additionally, with the help of MTS, the small-sized
Llama2-13b-chat substantially outperforms ChatGPT, facilitating an effective
deployment in real applications.

借助 ChatGPT 将写作熟练度分解成不同特征，并生成每个特征的评分标准，然后利用大型语言模型在几轮对话中提取特征得分，并通过特征平均和最大最小缩放得出总体得分，实验证明 Multi Trait Specialization 在所有大型语言模型和数据集上始终优于直接提示 (Vanilla)，同时在实际应用中，借助 MTS，小规模的 Llama2-13b-chat 显著优于 ChatGPT，实现有效的部署。