The state of the art in open-domain question answering (QA) relies on an
efficient retriever that drastically reduces the search space for the expensive
reader. A rather overlooked question in the community is the relationship
between the retriever and the reader, and in particular, if the whole purpose
of the retriever is just a fast approximation for the reader. Our empirical
evidence indicates that the answer is no, and that the reader and the retriever
are complementary to each other even in terms of accuracy only. We make a
careful conjecture that the architectural constraint of the retriever, which
has been originally intended for enabling approximate search, seems to also
make the model more robust in large-scale search. We then propose to distill
the reader into the retriever so that the retriever absorbs the strength of the
reader while keeping its own benefit. Experimental results show that our method
can enhance the document recall rate as well as the end-to-end QA accuracy of
off-the-shelf retrievers in open-domain QA tasks.

该研究讨论了基于开放域问题回答的现状，重点研究了 retriever 和 reader 之间的关系，通过实证分析表明，两者在准确性方面互为补充。其次提出将 reader 稀疏化，吸收 reader 的优势并保持自身的优点，实验结果表明我们的方法可以提高 off-the-shelf retrievers 在开放域 QA 任务中的文档查全率以及端到端 QA 的准确性。