Question answering over hybrid contexts is a complex task, which requires the
combination of information extracted from unstructured texts and structured
tables in various ways. Recently, In-Context Learning demonstrated significant
performance advances for reasoning tasks. In this paradigm, a large language
model performs predictions based on a small set of supporting exemplars. The
performance of In-Context Learning depends heavily on the selection procedure
of the supporting exemplars, particularly in the case of HybridQA, where
considering the diversity of reasoning chains and the large size of the hybrid
contexts becomes crucial. In this work, we present Selection of ExEmplars for
hybrid Reasoning (SEER), a novel method for selecting a set of exemplars that
is both representative and diverse. The key novelty of SEER is that it
formulates exemplar selection as a Knapsack Integer Linear Program. The
Knapsack framework provides the flexibility to incorporate diversity
constraints that prioritize exemplars with desirable attributes, and capacity
constraints that ensure that the prompt size respects the provided capacity
budgets. The effectiveness of SEER is demonstrated on FinQA and TAT-QA, two
real-world benchmarks for HybridQA, where it outperforms previous exemplar
selection methods.

SEER 是一种选择代表性和多样性的示例集合的新方法，通过将示例选择问题转化为背包整数线性规划问题，通过灵活调整多样性和容量约束条件来提高在 HybridQA 中的性能。在 FinQA 和 TAT-QA 两个真实世界的 HybridQA 基准测试中，SEER 优于先前的示例选择方法。

SEER：一种用于上下文混合问答的背包法示例选择

SEER: A Knapsack approach to Exemplar Selection for In-Context HybridQA

To facilitate conversational question answering (CQA) over hybrid contexts in
finance, we present a new dataset, named PACIFIC. Compared with existing CQA
datasets, PACIFIC exhibits three key features: (i) proactivity, (ii) numerical
reasoning, and (iii) hybrid context of tables and text. A new task is defined
accordingly to study Proactive Conversational Question Answering (PCQA), which
combines clarification question generation and CQA. In addition, we propose a
novel method, namely UniPCQA, to adapt a hybrid format of input and output
content in PCQA into the Seq2Seq problem, including the reformulation of the
numerical reasoning process as code generation. UniPCQA performs multi-task
learning over all sub-tasks in PCQA and incorporates a simple ensemble strategy
to alleviate the error propagation issue in the multi-task learning by
cross-validating top-$k$ sampled Seq2Seq outputs. We benchmark the PACIFIC
dataset with extensive baselines and provide comprehensive evaluations on each
sub-task of PCQA.

为促进在金融领域中的混合上下文中的会话式问答，我们提出了一个名为 PACIFIC 的新数据集，其中包含三个关键特征：积极性、数字推理和文本表格的混合上下文。我们定义了一个新的任务来研究积极型会话式问答 (PCQA)，该任务将澄清问题生成和 CQA 相结合。此外，我们提出了一种新的方法 UniPCQA，将 PCQA 中的输入输出内容混合格式转化为 Seq2Seq 模型问题，并将数字推理过程重新制定为代码生成形式。UniPCQA 通过对 PCQA 中所有子任务进行多任务学习，并采用简单的集成策略，在跨验证前 $k$ 个样本的 Seq2Seq 输出来缓解多任务学习中的错误传播问题。我们使用广泛的基线测试 PACIFIC 数据集，并对 PCQA 的每个子任务进行全面的评估。