Generating diverse sequences is important in many NLP applications such as
question generation or summarization that exhibit semantically one-to-many
relationships between source and the target sequences. We present a method to
explicitly separate diversification from generation using a general
plug-and-play module (called SELECTOR) that wraps around and guides an existing
encoder-decoder model. The diversification stage uses a mixture of experts to
sample different binary masks on the source sequence for diverse content
selection. The generation stage uses a standard encoder-decoder model given
each selected content from the source sequence. Due to the non-differentiable
nature of discrete sampling and the lack of ground truth labels for binary
mask, we leverage a proxy for ground truth mask and adopt stochastic hard-EM
for training. In question generation (SQuAD) and abstractive summarization
(CNN-DM), our method demonstrates significant improvements in accuracy,
diversity and training efficiency, including state-of-the-art top-1 accuracy in
both datasets, 6% gain in top-5 accuracy, and 3.7 times faster training over a
state of the art model. Our code is publicly available at
this https URL

该论文提出了一种使用 SELECTOR 模块来明确区分多样性和生成过程的方法，在 NLP 任务中证明了较高的准确性、多元性和训练效率。