The large language models have achieved superior performance on various
natural language tasks. One major drawback of such approaches is they are
resource-intensive in fine-tuning new datasets. Soft-prompt tuning presents a
resource-efficient solution to fine-tune the pre-trained language models (PLMs)
while keeping their weight frozen. Existing soft prompt methods mainly focus on
designing the input-independent prompts that steer the model to fit the domain
of the new dataset. Those methods often ignore the fine-grained information
about the task and context of the text. In this paper, we propose a multi-level
prompt tuning (MPrompt) method for machine reading comprehension. It utilizes
prompts at task-specific, domain-specific, and context-specific levels to
enhance the comprehension of input semantics at different granularities. We
also propose an independence constraint to steer each domain-specific prompt to
focus on information within its domain to avoid redundancy. Moreover, we
present a prompt generator that incorporates context-related knowledge in the
prompt generation to enhance contextual relevancy. We conducted extensive
experiments on 12 benchmarks of various QA formats and achieved an average
improvement of 1.94\% over the state-of-the-art methods.

该研究提出了一种多级提示调优方法来增强机器阅读理解，通过利用任务特定、领域特定和上下文相关的提示，提高了输入语义在不同层次上的理解能力。同时，引入了独立性约束来避免冗余，并利用上下文相关知识生成合适的提示。在各种问答格式的 12 个基准测试中进行了广泛实验，相比最先进的方法，平均改进率达到了 1.94%。