Providing knowledge documents for large language models (LLMs) has emerged as
a promising solution to update the static knowledge inherent in their
parameters. However, knowledge in the document may conflict with the memory of
LLMs due to outdated or incorrect knowledge in the LLMs' parameters. This leads
to the necessity of examining the capability of LLMs to assimilate supplemental
external knowledge that conflicts with their memory. While previous studies
have explained to what extent LLMs extract conflicting knowledge from the
provided text, they neglect the necessity to reason with conflicting knowledge.
Furthermore, there lack a detailed analysis on strategies to enable LLMs to
resolve conflicting knowledge via prompting, decoding strategy, and supervised
fine-tuning. To address these limitations, we construct a new dataset, dubbed
KNOT, for knowledge conflict resolution examination in the form of question
answering. KNOT facilitates in-depth analysis by dividing reasoning with
conflicting knowledge into three levels: (1) Direct Extraction, which directly
extracts conflicting knowledge to answer questions. (2) Explicit Reasoning,
which reasons with conflicting knowledge when the reasoning path is explicitly
provided in the question. (3) Implicit Reasoning, where reasoning with
conflicting knowledge requires LLMs to infer the reasoning path independently
to answer questions. We also conduct extensive experiments on KNOT to establish
empirical guidelines for LLMs to utilize conflicting knowledge in complex
circumstances. Dataset and associated codes can be accessed at
this https URL .

利用新数据集 KNOT 对大型语言模型进行与知识冲突处理相关的能力检验，将冲突知识的推理划分为三个层次，即直接提取、显式推理和隐式推理，并通过详细实验建立了利用冲突知识的指导原则。

解开纠纷：在大型语言模型中交织相互冲突的知识和推理技能

Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning  Skills in Large Language Models

This paper introduces a novel method for merging open-domain terminological
knowledge. It takes advantage of the Region Connection Calculus (RCC5), a
formalism used to represent regions in a topological space and to reason about
their set-theoretic relationships. To this end, we first propose a faithful
translation of terminological knowledge provided by several and potentially
conflicting sources into region spaces. The merging is then performed on these
spaces, and the result is translated back into the underlying language of the
input sources. Our approach allows us to benefit from the expressivity and the
flexibility of RCC5 while dealing with conflicting knowledge in a principled
way.

本文提出了一种新的方法来合并开放领域的术语知识，该方法利用了区域连接演算（RCC5）。我们首先将多个潜在冲突来源提供的术语知识信任地转化为区域空间，然后在这些空间上执行合并，并将结果转换回输入源的底层语言。我们的方法允许我们从 RCC5 的表达能力和灵活性中受益，同时以原则性的方式处理冲突知识。