Current research on generative language models (GLMs) for automated text
scoring (ATS) has focused almost exclusively on querying proprietary models via
Application Programming Interfaces (APIs). Yet such practices raise issues
around transparency and security, and these methods offer little in the way of
efficiency or customizability. With the recent proliferation of smaller,
open-source models, there is the option to explore GLMs with computers equipped
with modest, consumer-grade hardware, that is, for the "GPU poor." In this
study, we analyze the performance and efficiency of open-source, small-scale
GLMs for ATS. Results show that GLMs can be fine-tuned to achieve adequate,
though not state-of-the-art, performance. In addition to ATS, we take small
steps towards analyzing models' capacity for generating feedback by prompting
GLMs to explain their scores. Model-generated feedback shows promise, but
requires more rigorous evaluation focused on targeted use cases.

通过分析开源的小型生成语言模型（GLMs）在自动化文本评分（ATS）中的性能和效率，结果表明 GLMs 可以进行适当的微调以达到足够的性能。此外，在分析模型生成反馈能力方面，虽然由模型生成的反馈有潜力，但需要更严格的针对特定使用情况的评估。

基于生成型 AI 的自动文本评分技术在性能较低的 GPU 环境中的应用

Automated Text Scoring in the Age of Generative AI for the GPU-poor

Automated Text Scoring (ATS) provides a cost-effective and consistent
alternative to human marking. However, in order to achieve good performance,
the predictive features of the system need to be manually engineered by human
experts. We introduce a model that forms word representations by learning the
extent to which specific words contribute to the text's score. Using Long-Short
Term Memory networks to represent the meaning of texts, we demonstrate that a
fully automated framework is able to achieve excellent results over similar
approaches. In an attempt to make our results more interpretable, and inspired
by recent advances in visualizing neural networks, we introduce a novel method
for identifying the regions of the text that the model has found more
discriminative.

提出一种基于 Long-Short Term Memory 网络的自动化文本评分框架，该框架通过学习特定单词对文本评分的贡献来构建单词表示，并且能够在可视化神经网络的启发下，识别出文本中更具区分性的区域，以提高模型可解释性。