Open-domain question answering can be formulated as a phrase retrieval
problem, in which we can expect huge scalability and speed benefit but often
suffer from low accuracy due to the limitation of existing phrase
representation models. In this paper, we aim to improve the quality of each
phrase embedding by augmenting it with a contextualized sparse representation
(Sparc). Unlike previous sparse vectors that are term-frequency-based (e.g.,
tf-idf) or directly learned (only few thousand dimensions), we leverage
rectified self-attention to indirectly learn sparse vectors in n-gram
vocabulary space. By augmenting the previous phrase retrieval model (Seo et
al., 2019) with Sparc, we show 4%+ improvement in CuratedTREC and SQuAD-Open.
Our CuratedTREC score is even better than the best known retrieve & read model
with at least 45x faster inference speed.

本文旨在通过引入上下文稀疏表示（Sparc）来改善每个短语嵌入的质量，借助修正的自注意力间接地学习 N-gram 词汇空间中的稀疏向量，通过将前一短语检索模型（Seo 等人，2019）与 Sparc 相结合，我们在 CuratedTREC 和 SQuAD-Open 中展示了 4% 以上的改进，其中我们的 CuratedTREC 得分甚至优于已知的检索和阅读模型，而推理速度至少快 45 倍。