Previous studies have established that language models manifest stereotyped biases. Existing debiasing strategies, such as retraining a model with counterfactual data, representation projection, and prompting often fail to efficiently eliminate bias or directly alter the models' biased internal representations. To address these issues, we propose BiasEdit, an efficient model editing method to remove stereotypical bias from language models through lightweight networks that act as editors to generate parameter updates. BiasEdit employs a debiasing loss guiding editor networks to conduct local edits on partial parameters of a language model for debiasing while preserving the language modeling abilities during editing through a retention loss. Experiments on StereoSet and Crows-Pairs demonstrate the effectiveness, efficiency, and robustness of BiasEdit in eliminating bias compared to tangental debiasing baselines and little to no impact on the language models' general capabilities. In addition, we conduct bias tracing to probe bias in various modules and explore bias editing impacts on different components of language models.

本研究解决了语言模型中存在的刻板印象偏见问题，提出了BiasEdit这一高效的模型编辑方法。通过轻量网络生成参数更新，该方法有效地消除了偏见，同时保持了语言模型的语言生成能力。实验结果表明，BiasEdit在消除偏见方面表现优越，对模型整体能力影响较小。

BiasEdit：通过模型编辑消除刻板印象语言模型的偏见