Language models can store vast amounts of factual knowledge, but their
ability to use this knowledge for logical reasoning remains questionable. This
paper explores a language model's ability to manipulate its stored knowledge
during inference. We focus on four manipulation types: retrieval (e.g., "What
is person A's attribute X"), classification (e.g., "Is A's attribute X even or
odd?"), comparison (e.g., "Is A greater than B in attribute X?") and inverse
search (e.g., "Which person's attribute X equals T?")
We observe that pre-trained language models like GPT2/3/4 excel in knowledge
retrieval but struggle with simple classification or comparison tasks unless
Chain of Thoughts (CoTs) are employed during both training and inference. They
also perform poorly in inverse knowledge search, irrespective of the prompts.
Our primary contribution is a synthetic dataset for a controlled experiment
that confirms these inherent weaknesses: a language model cannot efficiently
manipulate knowledge from pre-training data, even when such knowledge is
perfectly stored and fully extractable in the models, and despite adequate
instruct fine-tuning.

本文研究了语言模型在推理过程中利用存储的知识的能力，发现预训练的语言模型在知识检索方面表现出色，但在简单的分类、比较和逆向搜索任务方面表现较差，即使在训练和推理过程中使用了控制思维链。研究的主要贡献是通过一个合成数据集的控制实验，证实了语言模型无法有效地操作预训练数据中的知识，即使这些知识完全存储且模型可以完全提取，且进行了适当的指导微调。