This position paper explores the rapid development of Foundation Models (FMs) in AI and their implications for intelligence and reasoning. It examines the characteristics of FMs, including their training on vast datasets and use of embedding spaces to capture semantic relationships. The paper discusses recent advancements in FMs' reasoning abilities which we argue cannot be attributed to increased model size but to novel training techniques which yield learning phenomena like grokking. It also addresses the challenges in benchmarking FMs and compares their structure to the human brain. We argue that while FMs show promising developments in reasoning and knowledge representation, understanding their inner workings remains a significant challenge, similar to ongoing efforts in neuroscience to comprehend human brain function. Despite having some similarities, fundamental differences between FMs and the structure of human brain warn us against making direct comparisons or expecting neuroscience to provide immediate insights into FM function.

本论文探讨了人工智能领域基础模型（FMs）的快速发展及其对智能与推理的影响。研究揭示了FMs的训练特点及其推理能力的最新进展，强调这些进展并非单纯因为模型规模的增加，而是源于新颖的训练技术。这项研究的主要发现是，尽管FMs在推理与知识表示方面展现出良好发展，但理解其内部机制仍然是一个重大挑战。 

理解基础模型：我们是否回到了1924年？