Conceptual reasoning, the ability to reason in abstract and high-level perspectives, is key to generalization in human cognition. However, limited study has been done on large language models' capability to perform conceptual reasoning. In this work, we bridge this gap and propose a novel conceptualization framework that forces models to perform conceptual reasoning on abstract questions and generate solutions in a verifiable symbolic space. Using this framework as an analytical tool, we show that existing large language models fall short on conceptual reasoning, dropping 9% to 28% on various benchmarks compared to direct inference methods. We then discuss how models can improve since high-level abstract reasoning is key to unbiased and generalizable decision-making. We propose two techniques to add trustworthy induction signals by generating familiar questions with similar underlying reasoning paths and asking models to perform self-refinement. Experiments show that our proposed techniques improve models' conceptual reasoning performance by 8% to 11%, achieving a more robust reasoning system that relies less on inductive biases.

我们提出了一个新的概念化框架，迫使模型在抽象问题上进行概念推理并在可验证的符号空间中生成解决方案，使用这个框架作为分析工具，我们发现现有的大型语言模型在概念推理方面存在不足，并通过引入可信的归纳信号来改善模型的概念推理性能，实验证明我们提出的技术使模型的概念推理性能提高了8%至11%，实现了一个更强大的推理系统，更少地依赖归纳偏见。

语言模型中的概念和公正推理