Since its launch in November 2022, ChatGPT has had a transformative effect on
education where students are using it to help with homework assignments and
teachers are actively employing it in their teaching practices. This includes
using ChatGPT as a tool for writing teachers to grade and generate feedback on
students' essays. In this study, we evaluated the quality of the feedback
generated by ChatGPT regarding the coherence and cohesion of the essays written
by English Language Learners (ELLs) students. We selected 50 argumentative
essays and generated feedback on coherence and cohesion using the ELLIPSE
rubric. During the feedback evaluation, we used a two-step approach: first,
each sentence in the feedback was classified into subtypes based on its
function (e.g., positive reinforcement, problem statement). Next, we evaluated
its accuracy and usability according to these types. Both the analysis of
feedback types and the evaluation of accuracy and usability revealed that most
feedback sentences were highly abstract and generic, failing to provide
concrete suggestions for improvement. The accuracy in detecting major problems,
such as repetitive ideas and the inaccurate use of cohesive devices, depended
on superficial linguistic features and was often incorrect. In conclusion,
ChatGPT, without specific training for the feedback generation task, does not
offer effective feedback on ELL students' coherence and cohesion.

ChatGPT 在无具体任务训练的情况下，对于英语学习者的文章连贯性和衔接性未提供有效反馈。