We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4 prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and GPT-3.5 (14%), but falling short of chance and the baseline set by human participants (63%). Participants' decisions were based mainly on linguistic style (35%) and socio-emotional traits (27%), supporting the idea that intelligence is not sufficient to pass the Turing Test. Participants' demographics, including education and familiarity with LLMs, did not predict detection rate, suggesting that even those who understand systems deeply and interact with them frequently may be susceptible to deception. Despite known limitations as a test of intelligence, we argue that the Turing Test continues to be relevant as an assessment of naturalistic communication and deception. AI models with the ability to masquerade as humans could have widespread societal consequences, and we analyse the effectiveness of different strategies and criteria for judging humanlikeness.

GPT-4在一个在线的图灵测试中表现出色，通过了41%的比赛，胜过了ELIZA（27%）和GPT-3.5（14%）的基准，但不及人类参与者（63%）的基准。参与者的决策主要基于语言风格（35%）和社交情感特征（27%），支持智能不足以通过图灵测试的观点。与教育程度和对LLMs的熟悉程度等参与者的个人信息无法预测检测率，这表明即使是那些深入了解系统并经常与其互动的人也可能受到欺骗。尽管图灵测试作为智能测试具有已知限制，我们认为它仍然是评估自然交流和欺骗的相关工具。具备冒充人类能力的AI模型可能对社会产生广泛的影响，我们分析了不同策略和准则来评判人类的相似性的效果。

GPT-4是否通过图灵测试？