We propose Masker, an unsupervised text-editing method for style transfer. To tackle cases when no parallel source-target pairs are available, we train masked language models (MLMs) for both the source and the target domain. Then we find the text spans where the two models disagree the most in terms of likelihood. This allows us to identify the source tokens to delete to transform the source text to match the style of the target domain. The deleted tokens are replaced with the target MLM, and by using a padded MLM variant, we avoid having to predetermine the number of inserted tokens. Our experiments on sentence fusion and sentiment transfer demonstrate that Masker performs competitively in a fully unsupervised setting. Moreover, in low-resource settings, it improves supervised methods' accuracy by over 10 percentage points when pre-training them on silver training data generated by Masker.

该论文提出了Masker，一种用于风格转移的无监督文本编辑方法，它利用掩码语言模型(Masked Language Models)在没有平行数据的情况下训练源语言和目标语言模型，并通过删除源文本的部分标记和替换为目标模型的标记，实现风格的转换。此方法在低资源环境中可以提高监督方法的准确性。

带填充掩码语言模型的无监督文本风格转移