Large Language Models (LLMs) have demonstrated good performance in many
reasoning tasks, but they still struggle with some complicated reasoning tasks
including logical reasoning. One non-negligible reason for LLMs' suboptimal
performance on logical reasoning is their overlooking of understanding logical
fallacies correctly. To evaluate LLMs' capability of logical fallacy
understanding (LFU), we propose five concrete tasks from three cognitive
dimensions of WHAT, WHY, and HOW in this paper. Towards these LFU tasks, we
have successfully constructed a new dataset LFUD based on GPT-4 accompanied by
a little human effort. Our extensive experiments justify that our LFUD can be
used not only to evaluate LLMs' LFU capability, but also to fine-tune LLMs to
obtain significantly enhanced performance on logical reasoning.

大型语言模型 (LLMs) 在很多推理任务中表现出良好的性能，但在某些复杂推理任务，特别是逻辑推理方面仍然存在困难。为了评估 LLMs 的逻辑谬误理解能力 (LFU)，我们在本文中从 WHAT、WHY 和 HOW 三个认知维度中提出了五个具体任务。为了解决这些 LFU 任务，我们成功构建了一个新的基于 GPT-4 的数据集 LFUD，只需少量人工参与。我们的广泛实验证明，我们的 LFUD 不仅可以用于评估 LLMs 的 LFU 能力，还可以通过微调 LLMs 在逻辑推理方面获得显著的性能提升。