Recent studies have revealed that the widely-used Pre-trained Language Models
(PLMs) propagate societal biases from the large unmoderated pre-training
corpora. Existing solutions require debiasing training processes and datasets
for debiasing, which are resource-intensive and costly. Furthermore, these
methods hurt the PLMs' performance on downstream tasks. In this study, we
propose Gender-tuning, which debiases the PLMs through fine-tuning on
downstream tasks' datasets. For this aim, Gender-tuning integrates Masked
Language Modeling (MLM) training objectives into fine-tuning's training
process. Comprehensive experiments show that Gender-tuning outperforms the
state-of-the-art baselines in terms of average gender bias scores in PLMs while
improving PLMs' performance on downstream tasks solely using the downstream
tasks' dataset. Also, Gender-tuning is a deployable debiasing tool for any PLM
that works with original fine-tuning.

通过使用下游任务数据对预训练语言模型（PLMs）进行微调， Gender-tuning 方法能够消除 PLMs 中的性别偏见，同时提高 PLMs 在下游任务上的性能。