Large language models (LLMs) have shown an impressive ability to perform
tasks believed to require thought processes. When the model does not document
an explicit thought process, it becomes difficult to understand the processes
occurring within its hidden layers and to determine if these processes can be
referred to as reasoning. We introduce a novel and interpretable analysis of
internal multi-hop reasoning processes in LLMs. We demonstrate that the
prediction process for compositional reasoning questions can be modeled using a
simple linear transformation between two semantic category spaces. We show that
during inference, the middle layers of the network generate highly
interpretable embeddings that represent a set of potential intermediate answers
for the multi-hop question. We use statistical analyses to show that a
corresponding subset of tokens is activated in the model's output, implying the
existence of parallel reasoning paths. These observations hold true even when
the model lacks the necessary knowledge to solve the task. Our findings can
help uncover the strategies that LLMs use to solve reasoning tasks, offering
insights into the types of thought processes that can emerge from artificial
intelligence. Finally, we also discuss the implication of cognitive modeling of
these results.

大型语言模型具有出色的能力来执行需要思考过程的任务，本研究引入了一种新颖且可解释的分析方法，探讨了大型语言模型内部的多跳推理过程，揭示了模型通过简单线性变换来建模组合推理问题的预测过程，并且发现网络的中间层生成高度可解释的嵌入，代表了多跳问题的一组潜在中间答案，这些观察结果揭示了平行推理路径的存在，即使模型缺少解决任务所需的知识，这些结果有助于揭示大型语言模型解决推理任务的策略，并提供了人工智能所能产生的思维过程类型的见解，最后还讨论了认知建模的这些结果的含义。