Recently, ChatGPT has emerged as a powerful NLP tool that can carry out
several tasks. However, the range of languages ChatGPT can handle remains
largely a mystery. In this work, we investigate ChatGPT's language
identification abilities. For this purpose, we compile Babel-670, a benchmark
comprising $670$ languages representing $23$ language families. Languages in
Babel-670 run the gamut between the very high-resource to the very low-resource
and are spoken in five continents. We then study ChatGPT's (both GPT-3.5 and
GPT-4) ability to (i) identify both language names and language codes (ii)
under both zero- and few-shot conditions (iii) with and without provision of
label set. When compared to smaller finetuned language identification tools, we
find that ChatGPT lags behind. Our empirical analysis shows the reality that
ChatGPT still resides in a state of potential enhancement before it can
sufficiently serve diverse communities.

最近，ChatGPT 作为一种强大的 NLP 工具出现了，它可以执行多项任务。然而，ChatGPT 可以处理的语言范围仍然是个谜。本文研究了 ChatGPT 的语言识别能力，为此我们编制了 Babel-670，其中包括 670 种代表 23 个语言系的语言。Babel-670 的语言涵盖了从资源丰富到资源稀缺的广泛范围，并且分布在五大洲。我们研究了 ChatGPT（包括 GPT-3.5 和 GPT-4）在零样本和少量样本条件下，有无标签集提供的情况下，识别语言名称和语言代码的能力。与较小规模的微调语言识别工具相比，我们发现 ChatGPT 落后于它们。我们的实证分析显示 ChatGPT 仍处于一个潜在增强的状态，尚不足以为不同社区提供足够的服务。