Warning: this paper contains model outputs exhibiting offensiveness and
biases. Recently pre-trained language models (PLMs) have prospered in various
natural language generation (NLG) tasks due to their ability to generate fairly
fluent text. Nevertheless, these models are observed to capture and reproduce
harmful contents in training corpora, typically toxic language and social
biases, raising severe moral issues. Prior works on ethical NLG tackle
detoxifying and debiasing separately, which is problematic since we find
debiased models still exhibit toxicity while detoxified ones even exacerbate
biases. To address such a challenge, we propose the first unified framework of
detoxifying and debiasing called UDDIA, which jointly formalizes these two
problems as rectifying the output space. We theoretically interpret our
framework as learning a text distribution mixing weighted attributes. Besides,
UDDIA conducts adaptive optimization of only a few parameters during decoding
based on a parameter-efficient tuning schema without any training data. This
leads to minimal generation quality loss and improved rectification performance
with acceptable computational cost. Experimental results demonstrate that
compared to several strong baselines, UDDIA achieves debiasing and detoxifying
simultaneously and better balances efficiency and effectiveness, taking a
further step towards practical ethical NLG.

通过提出第一种统一框架 UDDIA 来解决自然语言生成中的道德问题，该框架同时解決了去除偏见和毒性语言的问题，并提升了文本生成性能。