We equip a smaller Language Model to generalise to answering challenging
compositional questions that have not been seen in training. To do so we
propose a combination of multitask supervised pretraining on up to 93 tasks
designed to instill diverse reasoning abilities, and a dense retrieval system
that aims to retrieve a set of evidential paragraph fragments. Recent progress
in question-answering has been achieved either through prompting methods
against very large pretrained Language Models in zero or few-shot fashion, or
by fine-tuning smaller models, sometimes in conjunction with information
retrieval. We focus on the less explored question of the extent to which
zero-shot generalisation can be enabled in smaller models with retrieval
against a corpus within which sufficient information to answer a particular
question may not exist. We establish strong baselines in this setting for
diverse evaluation datasets (StrategyQA, CommonsenseQA, IIRC, DROP, Musique and
ARC-DA), and show that performance can be significantly improved by adding
retrieval-augmented training datasets which are designed to expose our models
to a variety of heuristic reasoning strategies such as weighing partial
evidence or ignoring an irrelevant context.

我们在本文中提出了一种通过多任务监督预训练和密集检索系统的组合来实现对具有挑战性的复合问题的泛化的方法，并且展示了通过添加用于训练的检索增强数据集可以显著提高模型的性能。