Reasoning ability is one of the most crucial capabilities of a foundation
model, signifying its capacity to address complex reasoning tasks.
Chain-of-Thought (CoT) technique is widely regarded as one of the effective
methods for enhancing the reasoning ability of foundation models and has
garnered significant attention. However, the reasoning process of CoT is
linear, step-by-step, similar to personal logical reasoning, suitable for
solving general and slightly complicated problems. On the contrary, the
thinking pattern of an expert owns two prominent characteristics that cannot be
handled appropriately in CoT, i.e., high-order multi-hop reasoning and
multimodal comparative judgement. Therefore, the core motivation of this paper
is transcending CoT to construct a reasoning paradigm that can think like an
expert. The hyperedge of a hypergraph could connect various vertices, making it
naturally suitable for modelling high-order relationships. Inspired by this,
this paper innovatively proposes a multimodal Hypergraph-of-Thought (HoT)
reasoning paradigm, which enables the foundation models to possess the
expert-level ability of high-order multi-hop reasoning and multimodal
comparative judgement. Specifically, a textual hypergraph-of-thought is
constructed utilizing triple as the primary thought to model higher-order
relationships, and a hyperedge-of-thought is generated through multi-hop
walking paths to achieve multi-hop inference. Furthermore, we devise a visual
hypergraph-of-thought to interact with the textual hypergraph-of-thought via
Cross-modal Co-Attention Graph Learning for multimodal comparative
verification. Experimentations on the ScienceQA benchmark demonstrate the
proposed HoT-based T5 outperforms CoT-based GPT3.5 and chatGPT, which is on par
with CoT-based GPT4 with a lower model size.

本文提出了一种基于多样图思维的类专家级超图的推理范式 (HoT)，使基础模型具备高阶多跳推理和多模态比较判断的专家级能力。实验结果表明，该方法在 ScienceQA 基准上优于基于 CoT 的 GPT3.5 和 chatGPT，并且与基于 CoT 的 GPT4 相当，模型大小更小。