Question Answering (QA) is a longstanding challenge in natural language
processing. Existing QA works mostly focus on specific question types,
knowledge domains, or reasoning skills. The specialty in QA research hinders
systems from modeling commonalities between tasks and generalization for wider
applications. To address this issue, we present ProQA, a unified QA paradigm
that solves various tasks through a single model. ProQA takes a unified
structural prompt as the bridge and improves the QA-centric ability by
structural prompt-based pre-training. Through a structurally designed
prompt-based input schema, ProQA concurrently models the knowledge
generalization for all QA tasks while keeping the knowledge customization for
every specific QA task. Furthermore, ProQA is pre-trained with structural
prompt-formatted large-scale synthesized corpus, which empowers the model with
the commonly-required QA ability. Experimental results on 11 QA benchmarks
demonstrate that ProQA consistently boosts performance on both full data
fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore,
ProQA exhibits strong ability in both continual learning and transfer learning
by taking the advantages of the structural prompt.

ProQA 是一个统一的 QA 范式，通过结构提示进行预训练，可以在所有 QA 任务中同时建模知识概括，同时保持每个特定 QA 任务的知识定制。实验证明其在 11 个 QA 基准上展示了强大的性能提升，并在连续学习和迁移学习等方面表现出强大的能力。

ProQA: 结构化提示基础预训练用于统一问答

ProQA: Structural Prompt-based Pre-training for Unified Question Answering

In order to learn quickly with few samples, meta-learning utilizes prior
knowledge learned from previous tasks. However, a critical challenge in
meta-learning is task uncertainty and heterogeneity, which can not be handled
via globally sharing knowledge among tasks. In this paper, based on
gradient-based meta-learning, we propose a hierarchically structured
meta-learning (HSML) algorithm that explicitly tailors the transferable
knowledge to different clusters of tasks. Inspired by the way human beings
organize knowledge, we resort to a hierarchical task clustering structure to
cluster tasks. As a result, the proposed approach not only addresses the
challenge via the knowledge customization to different clusters of tasks, but
also preserves knowledge generalization among a cluster of similar tasks. To
tackle the changing of task relationship, in addition, we extend the
hierarchical structure to a continual learning environment. The experimental
results show that our approach can achieve state-of-the-art performance in both
toy-regression and few-shot image classification problems.

本文提出了一种基于梯度的分层结构的元学习方法，该方法通过任务聚类结构以及自适应知识传递来解决元学习中任务不确定性和异质性问题，并在玩具回归和少样本图像分类问题中取得了最优表现。