We introduce the Variational Open-Domain (VOD) framework for end-to-end
training and evaluation of retrieval-augmented models (open-domain question
answering and language modelling). We show that the Rényi variational bound,
a lower bound to the task marginal likelihood, can be exploited to aid
optimization and use importance sampling to estimate the task log-likelihood
lower bound and its gradients using samples drawn from an auxiliary retriever
(approximate posterior). The framework can be used to train modern
retrieval-augmented systems end-to-end using tractable and consistent estimates
of the Rényi variational bound and its gradients. We demonstrate the
framework's versatility by training reader-retriever BERT-based models on
multiple-choice medical exam questions (MedMCQA and USMLE). We registered a new
state-of-the-art for both datasets (MedMCQA: $62.9$\%, USMLE: $55.0$\%). Last,
we show that the retriever part of the learned reader-retriever model trained
on the medical board exam questions can be used in search engines for a medical
knowledge base.

本文介绍了 Variational Open-Domain（VOD）框架，用于训练和评估检索增强模型（开放领域问答和语言建模），研究了 Rényi 变分界限，使用重要性采样从辅助检索器抽取样本，并利用该框架在多项选择医学考试问题上训练了读者 - 检索器 BERT 模型。