Large language models (LLMs) have recently reached an impressive level of linguistic capability, prompting comparisons with human language skills. However, there have been relatively few systematic inquiries into the linguistic capabilities of the latest generation of LLMs, and those studies that do exist (i) ignore the remarkable ability of humans to generalize, (ii) focus only on English, and (iii) investigate syntax or semantics and overlook other capabilities that lie at the heart of human language, like morphology. Here, we close these gaps by conducting the first rigorous analysis of the morphological capabilities of ChatGPT in four typologically varied languages (specifically, English, German, Tamil, and Turkish). We apply a version of Berko's (1958) wug test to ChatGPT, using novel, uncontaminated datasets for the four examined languages. We find that ChatGPT massively underperforms purpose-built systems, particularly in English. Overall, our results -- through the lens of morphology -- cast a new light on the linguistic capabilities of ChatGPT, suggesting that claims of human-like language skills are premature and misleading.

大型语言模型 (LLMs) 的语言能力令人印象深刻，但在对其能力进行系统研究方面尚有较少的探讨。本研究通过对四种不同类型的语言 (英语、德语、泰米尔语和土耳其语) 进行首次严格分析 ChatGPT 的形态能力，发现 ChatGPT 在各语言中的性能远低于专门构建的系统，尤其在英语中表现不佳。总的来说，基于形态学的研究结果对 ChatGPT 的语言能力提出了新的看法，并暗示自称具有类人语言技能的说法为时尚早和误导性。

一个对于大型语言模型形态学能力的多语言调查：计算ChatGPT中Wugs的错误