Large Language Models have taken the cognitive science world by storm. It is
perhaps timely now to take stock of the various research paradigms that have
been used to make scientific inferences about ``cognition" in these models or
about human cognition. We review several emerging research paradigms --
GPT-ology, LLMs-as-computational-models, and ``silicon sampling" -- and review
recent papers that have used LLMs under these paradigms. In doing so, we
discuss their claims as well as challenges to scientific inference under these
various paradigms. We highlight several outstanding issues about LLMs that have
to be addressed to push our science forward: closed-source vs open-sourced
models; (the lack of visibility of) training data; and reproducibility in LLM
research, including forming conventions on new task ``hyperparameters" like
instructions and prompts.

大型语言模型在认知科学领域引起了轰动，现在是时候对用于在这些模型或人类认知中进行科学推理的各种研究范式进行总结。我们回顾了几种新兴的研究范式 - GPT-ology、LLMs 作为计算模型以及 “硅采样”，并回顾了最近在这些范式下使用 LLMs 的论文。在这样做的过程中，我们讨论了它们的主张以及在这些不同范式下进行科学推理的挑战。我们强调了几个关于 LLMs 需要解决的重要问题，以推动我们的科学发展：闭源 vs 开源模型；（缺乏的）训练数据；并且在 LLM 研究中的可重复性，包括形成新任务 “超参数”（如指令和提示）的约定。