We present a Chain-of-Action (CoA) framework for multimodal and
retrieval-augmented Question-Answering (QA). Compared to the literature, CoA
overcomes two major challenges of current QA applications: (i) unfaithful
hallucination that is inconsistent with real-time or domain facts and (ii) weak
reasoning performance over compositional information. Our key contribution is a
novel reasoning-retrieval mechanism that decomposes a complex question into a
reasoning chain via systematic prompting and pre-designed actions.
Methodologically, we propose three types of domain-adaptable `Plug-and-Play'
actions for retrieving real-time information from heterogeneous sources. We
also propose a multi-reference faith score (MRFS) to verify and resolve
conflicts in the answers. Empirically, we exploit both public benchmarks and a
Web3 case study to demonstrate the capability of CoA over other methods.

我们提出了一种多模态且具有检索增强的问题回答的行动链框架，通过系统提示和预设计的行动将复杂问题分解为推理链。我们使用三种可调整领域的 “即插即用” 操作来从异构来源中检索实时信息，并提出了一个多参考信任分数（MRFS）来验证和解决答案中的冲突。通过公共基准测试和 Web3 案例研究，我们验证了 CoA 相对于其他方法的能力。