Despite advancements in LLMs, knowledge-based reasoning remains a
longstanding issue due to the fragility of knowledge recall and inference.
Existing methods primarily encourage LLMs to autonomously plan and solve
problems or to extensively sample reasoning chains without addressing the
conceptual and inferential fallacies. Attempting to alleviate inferential
fallacies and drawing inspiration from multi-agent collaboration, we present a
framework to increase faithfulness and causality for knowledge-based reasoning.
Specifically, we propose to employ multiple intelligent agents (i.e., reasoner
and causal evaluator) to work collaboratively in a reasoning-and-consensus
paradigm for elevated reasoning faithfulness. The reasoners focus on providing
solutions with human-like causality to solve open-domain problems. On the other
hand, the causal evaluator agent scrutinizes if the answer in a solution is
causally deducible from the question and vice versa, with a counterfactual
answer replacing the original. According to the extensive and comprehensive
evaluations on a variety of knowledge reasoning tasks (e.g., science question
answering and commonsense reasoning), our framework outperforms all compared
state-of-the-art approaches by large margins.

基于多智能体协作的框架提高知识推理的真实性和因果性，并在各类知识推理任务中优于其他最先进方法。