We present two novel unsupervised methods for eliminating toxicity in text.
Our first method combines two recent ideas: (1) guidance of the generation
process with small style-conditional language models and (2) use of
paraphrasing models to perform style transfer. We use a well-performing
paraphraser guided by style-trained language models to keep the text content
and remove toxicity. Our second method uses BERT to replace toxic words with
their non-offensive synonyms. We make the method more flexible by enabling BERT
to replace mask tokens with a variable number of words. Finally, we present the
first large-scale comparative study of style transfer models on the task of
toxicity removal. We compare our models with a number of methods for style
transfer. The models are evaluated in a reference-free way using a combination
of unsupervised style transfer metrics. Both methods we suggest yield new SOTA
results.

本文提出了两种新颖的无监督方法来消除文本中的有害信息，第一种方法结合小型样式条件语言模型的生成过程指导和重新表述模型执行样式转移的想法，第二种方法利用 BERT 将有害词汇替换为它们的无冒犯同义词，并进行了大规模比较研究，证明这两种方法有效且是新的最佳解决方案。