In this report, we pose the following question: Who is the most intelligent
AI model to date, as measured by the OlympicArena (an Olympic-level,
multi-discipline, multi-modal benchmark for superintelligent AI)? We
specifically focus on the most recently released models: Claude-3.5-Sonnet,
Gemini-1.5-Pro, and GPT-4o. For the first time, we propose using an Olympic
medal Table approach to rank AI models based on their comprehensive performance
across various disciplines. Empirical results reveal: (1) Claude-3.5-Sonnet
shows highly competitive overall performance over GPT-4o, even surpassing
GPT-4o on a few subjects (i.e., Physics, Chemistry, and Biology). (2)
Gemini-1.5-Pro and GPT-4V are ranked consecutively just behind GPT-4o and
Claude-3.5-Sonnet, but with a clear performance gap between them. (3) The
performance of AI models from the open-source community significantly lags
behind these proprietary models. (4) The performance of these models on this
benchmark has been less than satisfactory, indicating that we still have a long
way to go before achieving superintelligence. We remain committed to
continuously tracking and evaluating the performance of the latest powerful
models on this benchmark (available at
this https URL).

谁是迄今为止最智能的人工智能模型？我们使用奥林匹克竞技场（一个超智能人工智能的奥林匹克级多学科多模式基准）进行评估，并提出使用奥林匹克奖牌榜来排名人工智能模型。实证结果表明：Claude-3.5-Sonnet 在各个学科具有竞争力的综合表现优于 GPT-4o，超过 GPT-4o 在物理、化学和生物学等学科。Gemini-1.5-Pro 和 GPT-4V 排名次于 GPT-4o 和 Claude-3.5-Sonnet，但它们之间存在明显的性能差距。开源社区的人工智能模型性能明显落后于专有模型。这些模型在这个基准上的表现不尽人意，表明我们在实现超智能之前还有很长的路要走。我们致力于持续追踪和评估最新强大模型在这个基准上的表现。