Retriever-reader models achieve competitive performance across many different NLP tasks such as open question answering and dialogue conversations. In this work, we notice these models easily overfit the top-rank retrieval passages and standard training fails to reason over the entire retrieval passages. We introduce a learnable passage mask mechanism which desensitizes the impact from the top-rank retrieval passages and prevents the model from overfitting. Controlling the gradient variance with fewer mask candidates and selecting the mask candidates with one-shot bi-level optimization, our learnable regularization strategy enforces the answer generation to focus on the entire retrieval passages. Experiments on different tasks across open question answering, dialogue conversation, and fact verification show that our method consistently outperforms its baselines. Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.

通过引入可学习的段落掩码机制，可以防止retriever-reader模型过拟合排名前几个检索段落，进而有效地解决NLP任务中针对整个检索段落推理的问题，并在开放式问题回答，对话交互和事实验证等不同任务中取得了胜过其他方法的性能表现。

Passage-Mask: 检索-阅读模型的可学习正则化策略