We propose a method to control the attributes of Language Models (LMs) for the text generation task using Causal Average Treatment Effect (ATE) scores and counterfactual augmentation. We explore this method, in the context of LM detoxification, and propose the Causally Fair Language (CFL) architecture for detoxifying pre-trained LMs in a plug-and-play manner. Our architecture is based on a Structural Causal Model (SCM) that is mathematically transparent and computationally efficient as compared with many existing detoxification techniques. We also propose several new metrics that aim to better understand the behaviour of LMs in the context of toxic text generation. Further, we achieve state of the art performance for toxic degeneration, which are computed using \RTP (RTP) benchmark. Our experiments show that CFL achieves such a detoxification without much impact on the model perplexity. We also show that CFL mitigates the unintended bias problem through experiments on the BOLD dataset.

使用因果平均处理效应（ATE）分数和反事实增强作为文本生成任务语言模型（LMs）属性控制的方法，我们提出了因果公平语言（CFL）架构，以插入并播放的方式解毒预训练LMs。我们的实验表明，CFL实现了这种解毒而不会对模型困惑度产生太大影响，并通过对BOLD数据集的实验表明，CFL可以缓解意外偏见问题。

通过令牌级属性控制生成实现因果公平的语言模型