ChatGPT is a publicly available chatbot that can quickly generate texts on
given topics, but it is unknown whether the chatbot is really superior to human
writers in all aspects of writing and whether its writing quality can be
prominently improved on the basis of updating commands. Consequently, this
study compared the writing performance on a narrative topic by ChatGPT and
Chinese intermediate English (CIE) learners so as to reveal the chatbot's
advantage and disadvantage in writing. The data were analyzed in terms of five
discourse components using Coh-Metrix (a special instrument for analyzing
language discourses), and the results revealed that ChatGPT performed better
than human writers in narrativity, word concreteness, and referential cohesion,
but worse in syntactic simplicity and deep cohesion in its initial version.
After more revision commands were updated, while the resulting version was
facilitated in syntactic simplicity, yet it is still lagged far behind CIE
learners' writing in deep cohesion. In addition, the correlation analysis of
the discourse components suggests that narrativity was correlated with
referential cohesion in both ChatGPT and human writers, but the correlations
varied within each group.

本研究通过 Coh-Metrix 比较了 ChatGPT 和 CIE 学生在叙事主题上的写作表现，结果表明，在初始版本中，ChatGPT 在叙述性、词汇具体性和指代连贯性方面优于人类作家，但在句法简单性和深度连贯性方面劣于人类作家。经过多次修订后，ChatGPT 在句法简单性方面得到了改善，但在深度连贯性方面仍远不如 CIE 学生的写作。另外，讨论组成分的相关性分析表明，ChatGPT 和人类作家的叙事性与指代连贯性呈正相关，但各组内的相关性不同。