Given unstructured text, Large Language Models (LLMs) are adept at answering
simple (single-hop) questions. However, as the complexity of the questions
increase, the performance of LLMs degrade. We believe this is due to the
overhead associated with understanding the complex question followed by
filtering and aggregating unstructured information in the raw text. Recent
methods try to reduce this burden by integrating structured knowledge triples
into the raw text, aiming to provide a structured overview that simplifies
information processing. However, this simplistic approach is query-agnostic and
the extracted facts are ambiguous as they lack context. To address these
drawbacks and to enable LLMs to answer complex (multi-hop) questions with ease,
we propose to use a knowledge graph (KG) that is context-aware and is distilled
to contain query-relevant information. The use of our compressed distilled KG
as input to the LLM results in our method utilizing up to $67\%$ fewer tokens
to represent the query relevant information present in the supporting
documents, compared to the state-of-the-art (SoTA) method. Our experiments show
consistent improvements over the SoTA across several metrics (EM, F1,
BERTScore, and Human Eval) on two popular benchmark datasets (HotpotQA and
MuSiQue).

利用压缩的知识图作为输入，我们的方法在使用较少标记表示支持文档中与查询相关信息方面比现有方法利用高达 67% 的标记。我们的实验证明，在两个流行的基准数据集（HotpotQA 和 MuSiQue）上，我们的方法在多个指标（EM，F1，BERTScore 和人类评估）上持续改进。