Question answering over heterogeneous data requires reasoning over diverse
sources of data, which is challenging due to the large scale of information and
organic coupling of heterogeneous data. Various approaches have been proposed
to address these challenges. One approach involves training specialized
retrievers to select relevant information, thereby reducing the input length.
Another approach is to transform diverse modalities of data into a single
modality, simplifying the task difficulty and enabling more straightforward
processing. In this paper, we propose HProPro, a novel program-based prompting
framework for the hybrid question answering task. HProPro follows the code
generation and execution paradigm. In addition, HProPro integrates various
functions to tackle the hybrid reasoning scenario. Specifically, HProPro
contains function declaration and function implementation to perform hybrid
information-seeking over data from various sources and modalities, which
enables reasoning over such data without training specialized retrievers or
performing modal transformations. Experimental results on two typical hybrid
question answering benchmarks HybridQA and MultiModalQA demonstrate the
effectiveness of HProPro: it surpasses all baseline systems and achieves the
best performances in the few-shot settings on both datasets.

我们提出了一种基于程序的提示框架 HProPro，用于处理混合问答任务，该框架通过整合各种功能来解决异构数据的混合推理场景，从而能够在没有训练专门的检索器或执行模态转换的情况下，对来自各种源和模态的数据进行推理。实验证明 HProPro 在两个典型的混合问答基准测试（HybridQA 和 MultiModalQA）上效果显著，超过了所有基准系统，在少样本设置下表现最佳。