Large language models (LLMs) have been widely used in various applications but are known to suffer from issues related to untruthfulness and toxicity. While parameter-efficient modules (PEMs) have demonstrated their effectiveness in equipping models with new skills, leveraging PEMs for deficiency unlearning remains underexplored. In this work, we propose a PEMs operation approach, namely Extraction-before-Subtraction (Ext-Sub), to enhance the truthfulness and detoxification of LLMs through the integration of ``expert'' PEM and ``anti-expert'' PEM. Remarkably, even anti-expert PEM possess valuable capabilities due to their proficiency in generating fabricated content, which necessitates language modeling and logical narrative competence. Rather than merely negating the parameters, our approach involves extracting and eliminating solely the deficiency capability within anti-expert PEM while preserving the general capabilities. To evaluate the effectiveness of our approach in terms of truthfulness and detoxification, we conduct extensive experiments on LLMs, encompassing additional abilities such as language modeling and mathematical reasoning. Our empirical results demonstrate that our approach effectively improves truthfulness and detoxification, while largely preserving the fundamental abilities of LLMs.

通过整合“专家”和“反专家”参数，我们提出了一种称为“Ext-Sub”的参数有效模块操作方法，以提高大型语言模型的真实性和去毒性，并在保留通用能力的同时提取和消除“反专家”参数内的缺陷能力。通过对语言模型和数学推理等额外能力进行广泛实验，我们的实证结果表明我们的方法有效地改善了大型语言模型的真实性和去毒性。

鉴真伪：通过高效参数模块操作进行模型缺陷遗忘