We present the first experiments on Native Language Identification (NLI)
using LLMs such as GPT-4. NLI is the task of predicting a writer's first
language by analyzing their writings in a second language, and is used in
second language acquisition and forensic linguistics. Our results show that GPT
models are proficient at NLI classification, with GPT-4 setting a new
performance record of 91.7% on the benchmark TOEFL11 test set in a zero-shot
setting. We also show that unlike previous fully-supervised settings, LLMs can
perform NLI without being limited to a set of known classes, which has
practical implications for real-world applications. Finally, we also show that
LLMs can provide justification for their choices, providing reasoning based on
spelling errors, syntactic patterns, and usage of directly translated
linguistic patterns.

使用 LLMs（如 GPT-4）进行原生语言识别（NLI）的实验结果表明，GPT 模型在 NLI 分类上表现出色，在零样本设置下取得了 91.7％的性能记录。与以往的完全监督设置不同，LLMs 可以在无需限制于已知类别的情况下执行 NLI，这在实际应用中具有实际意义。此外，LLMs 还能提供选择的理由，根据拼写错误、句法模式和直接翻译的语言模式进行推理。