Large language models (LLMs) have demonstrated impressive zero-shot or
few-shot commonsense reasoning performance on various natural language
processing (NLP) tasks. However, despite their strong commonsense reasoning
abilities, LLMs still exhibit various kinds of inconsistency problems. While
previous researches mainly focused on the self-consistency within a single LLM,
we propose to explore the inter-consistency issue between two or more LLMs,
which is critical for diverse and precise decision-making processes. Since the
LLMs possess human-like intelligence after instruction tuning and reinforcement
learning with human feedback (RLHF), we design a formal debate framework to
delve into the inter-consistency problem among LLMs with three-stage debate:
fair debate, mismatched debate, and roundtable debate. Through extensive
experiments on 7 commonsense reasoning datasets, LLMs not only become more
inter-consistent by compromising and refuting but also achieve higher
performance and stronger interpretability. Furthermore, we find a much stronger
LLM would be dominant in mismatched debates, while it will be easily misled by
relatively weaker LLMs in a more complex debate scenario such as roundtable
debate.

在人工智能的领域中，大型语言模型（LLMs）需求渐趋增长。本研究通过命题竞辩框架来探讨多个 LLMs 之间的持续性以及一致性问题，确立了公正比赛，失调比赛和圆桌比赛的三个阶段，并在多个常识推理数据集上进行了广泛的实验，从而提出了令人信服的解决方案