Paraphrase generation has benefited extensively from recent progress in the
designing of training objectives and model architectures. However, previous
explorations have largely focused on supervised methods, which require a large
amount of labeled data that is costly to collect. To address this drawback, we
adopt a transfer learning approach and propose a training pipeline that enables
pre-trained language models to generate high-quality paraphrases in an
unsupervised setting. Our recipe consists of task-adaptation, self-supervision,
and a novel decoding algorithm named Dynamic Blocking (DB). To enforce a
surface form dissimilar from the input, whenever the language model emits a
token contained in the source sequence, DB prevents the model from outputting
the subsequent source token for the next generation step. We show with
automatic and human evaluations that our approach achieves state-of-the-art
performance on both the Quora Question Pair (QQP) and the ParaNMT datasets and
is robust to domain shift between the two datasets of distinct distributions.
We also demonstrate that our model transfers to paraphrasing in other languages
without any additional finetuning.

本文研究了基于转移学习的无监督方法生成高质量的同义改写，其中采用了任务自适应、自监督学习和名为 Dynamic Blocking 的新型解码算法。该方法在问答数据集和 ParaNMT 数据集上均取得了最新的性能，并且在不同语言改写中具有良好的迁移性能。