Large Language Models have taken the cognitive science world by storm. It is perhaps timely now to take stock of the various research paradigms that have been used to make scientific inferences about ``cognition" in these models or about human cognition. We review several emerging research paradigms -- GPT-ology, LLMs-as-computational-models, and ``silicon sampling" -- and review recent papers that have used LLMs under these paradigms. In doing so, we discuss their claims as well as challenges to scientific inference under these various paradigms. We highlight several outstanding issues about LLMs that have to be addressed to push our science forward: closed-source vs open-sourced models; (the lack of visibility of) training data; and reproducibility in LLM research, including forming conventions on new task ``hyperparameters" like instructions and prompts.

大型语言模型在认知科学领域引起了轰动，现在是时候对用于在这些模型或人类认知中进行科学推理的各种研究范式进行总结。我们回顾了几种新兴的研究范式- GPT-ology、LLMs作为计算模型以及“硅采样”，并回顾了最近在这些范式下使用LLMs的论文。在这样做的过程中，我们讨论了它们的主张以及在这些不同范式下进行科学推理的挑战。我们强调了几个关于LLMs需要解决的重要问题，以推动我们的科学发展: 闭源 vs 开源模型;（缺乏的）训练数据; 并且在LLM研究中的可重复性, 包括形成新任务“超参数”（如指令和提示）的约定。

GPT-ology，计算模型，硅采样：认知科学中的LLMs如何思考？