Despite the remarkable performance of Large Language Models (LLMs), they still struggle with generating logically sound arguments, resulting in potential risks such as spreading misinformation. An important factor contributing to LLMs' suboptimal performance in generating coherent arguments is their oversight of logical fallacies. To address this issue, we introduce FIPO, a fallacy-informed framework that leverages preference optimization methods to steer LLMs toward logically sound arguments. FIPO includes a classification loss, to capture the fine-grained information on fallacy categories. Our results on argumentation datasets show that our method reduces the fallacy errors by up to 17.5%. Furthermore, our human evaluation results indicate that the quality of the generated arguments by our method significantly outperforms the fine-tuned baselines, as well as prior preference optimization methods, such as DPO. These findings highlight the importance of ensuring models are aware of logical fallacies for effective argument generation.

该研究解决了大型语言模型在生成逻辑上合理的论证时所面临的挑战，特别是对逻辑谬误的忽视。提出的FIPO框架通过偏好优化方法引导LLMs生成逻辑合理的论证，结果表明该方法能将谬误减少多达17.5%，显著提高了生成论证的质量。

基于逻辑谬误的论证生成框架