Recent advances in open-domain question answering (ODQA) have demonstrated impressive accuracy on standard Wikipedia style benchmarks. However, it is less clear how robust these models are and how well they perform when applied to real-world applications in drastically different domains. While there has been some work investigating how well ODQA models perform when tested for out-of-domain (OOD) generalization, these studies have been conducted only under conservative shifts in data distribution and typically focus on a single component (ie. retrieval) rather than an end-to-end system. In response, we propose a more realistic and challenging domain shift evaluation setting and, through extensive experiments, study end-to-end model performance. We find that not only do models fail to generalize, but high retrieval scores often still yield poor answer prediction accuracy. We then categorize different types of shifts and propose techniques that, when presented with a new dataset, predict if intervention methods are likely to be successful. Finally, using insights from this analysis, we propose and evaluate several intervention methods which improve end-to-end answer F1 score by up to 24 points.

研究开放领域问答（ODQA）中，当应用于广泛不同的领域时，此类模型的稳健性和应用性能。该研究提出了一个更现实和具有挑战性的领域转移评估环境，并研究了端到端的模型性能。他们发现，不仅模型在推广方面表现出失败，且高检索分数通常也不能提供准确的答案预测。最后，研究提出并评估了多种干预方法，其提高了端到端答案F1得分，最多可达24个点。

开放领域问答中领域自适应挑战与干预：调整或注释？