Dual encoders have been used for question-answering (QA) and information
retrieval (IR) tasks with good results. Previous research focuses on two major
types of dual encoders, Siamese Dual Encoder (SDE), with parameters shared
across two encoders, and Asymmetric Dual Encoder (ADE), with two distinctly
parameterized encoders. In this work, we explore different ways in which the
dual encoder can be structured, and evaluate how these differences can affect
their efficacy in terms of QA retrieval tasks. By evaluating on MS MARCO, open
domain NQ and the MultiReQA benchmarks, we show that SDE performs significantly
better than ADE. We further propose three different improved versions of ADEs
by sharing or freezing parts of the architectures between two encoder towers.
We find that sharing parameters in projection layers would enable ADEs to
perform competitively with or outperform SDEs. We further explore and explain
why parameter sharing in projection layer significantly improves the efficacy
of the dual encoders, by directly probing the embedding spaces of the two
encoder towers with t-SNE algorithm.

通过对 SDE 和 ADE 的探究与对比实验，我们发现共享参数可以有效提高 ADE 在 QA 检索任务中的性能，甚至超过 SDE，并进一步提出了三种不同的 ADE 改进版本。