LLMs usually exhibit limitations in their ability to incorporate new
knowledge, the generation of hallucinations, and the transparency of their
decision-making process. In this paper, we explore how to prompt LLMs with
knowledge graphs (KG), working as a remedy to engage LLMs with up-to-date
knowledge and elicit the reasoning pathways from LLMs. Specifically, we build a
prompting pipeline that endows LLMs with the capability of comprehending KG
inputs and inferring with a combined implicit knowledge and the retrieved
external knowledge. In addition, we investigate eliciting the mind map on which
LLMs perform the reasoning and generate the answers. It is identified that the
produced mind map exhibits the reasoning pathways of LLMs grounded on the
ontology of knowledge, hence bringing the prospects of probing and gauging LLM
inference in production. The experiments on three question & answering datasets
also show that MindMap prompting leads to a striking empirical gain. For
instance, prompting a GPT-3.5 with MindMap yields an overwhelming performance
over GPT-4 consistently. We also demonstrate that with structured facts
retrieved from KG, MindMap can outperform a series of
prompting-with-document-retrieval methods, benefiting from more accurate,
concise, and comprehensive knowledge from KGs.

借助知识图谱，本研究探索了如何通过知识图谱向 LLMs 提供新知识和推理路径，并通过生成心智地图来解释 LLMs 的推理过程，实验证明 MindMap 方法在问答任务中显著提升了性能。

思维导图：大型语言模型中的知识图谱引发思维火花

MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large  Language Models

Multiple critical scenarios (like Wikipedia text generation given English
Infoboxes) need automated generation of descriptive text in low resource (LR)
languages from English fact triples. Previous work has focused on English
fact-to-text (F2T) generation. To the best of our knowledge, there has been no
previous attempt on cross-lingual alignment or generation for LR languages.
Building an effective cross-lingual F2T (XF2T) system requires alignment
between English structured facts and LR sentences. We propose two unsupervised
methods for cross-lingual alignment. We contribute XALIGN, an XF2T dataset with
0.45M pairs across 8 languages, of which 5402 pairs have been manually
annotated. We also train strong baseline XF2T generation models on the XAlign
dataset.

提出两种无监督对齐方法，建立英语结构化的事实与低资源语言句子的有效交叉生成，构建了包含 0.45M 对跨越 8 种语言的 XAlign 数据集，其中有 5402 种对进行了手动标注，并在该数据集上训练了强大的基线 XF2T 生成模型。

XAlign: 面向低资源语言的跨语言事实到文本对齐与生成

XAlign: Cross-lingual Fact-to-Text Alignment and Generation for Low-Resource Languages

Learning how to predict future events from patterns of past events is
difficult when the set of possible event types is large. Training an
unrestricted neural model might overfit to spurious patterns. To exploit
domain-specific knowledge of how past events might affect an event's present
probability, we propose using a temporal deductive database to track structured
facts over time. Rules serve to prove facts from other facts and from past
events. Each fact has a time-varying state---a vector computed by a neural net
whose topology is determined by the fact's provenance, including its experience
of past events. The possible event types at any time are given by special
facts, whose probabilities are neurally modeled alongside their states. In both
synthetic and real-world domains, we show that neural probabilistic models
derived from concise Datalog programs improve prediction by encoding
appropriate domain knowledge in their architecture.

利用时态推理数据库跟踪结构化事实来探索过去事件如何影响事件的现有概率，并通过神经概率模型来提高预测准确率。

通过时间的神经 Datalog：通过逻辑规范实现知情的时间建模

Neural Datalog Through Time: Informed Temporal Modeling via Logical  Specification

We study scalable and uniform understanding of facts in images. Existing
visual recognition systems are typically modeled differently for each fact type
such as objects, actions, and interactions. We propose a setting where all
these facts can be modeled simultaneously with a capacity to understand
unbounded number of facts in a structured way. The training data comes as
structured facts in images, including (1) objects (e.g., $<$boy$>$), (2)
attributes (e.g., $<$boy, tall$>$), (3) actions (e.g., $<$boy, playing$>$), and
(4) interactions (e.g., $<$boy, riding, a horse $>$). Each fact has a semantic
language view (e.g., $<$ boy, playing$>$) and a visual view (an image with this
fact). We show that learning visual facts in a structured way enables not only
a uniform but also generalizable visual understanding. We propose and
investigate recent and strong approaches from the multiview learning literature
and also introduce two learning representation models as potential baselines.
We applied the investigated methods on several datasets that we augmented with
structured facts and a large scale dataset of more than 202,000 facts and
814,000 images. Our experiments show the advantage of relating facts by the
structure by the proposed models compared to the designed baselines on
bidirectional fact retrieval.

本研究提出了在图像中可扩展和统一地理解事实的方案，通过学习有结构的视觉事实，实现了一种均一且具有通用性的视觉理解，并在拓展了结构化事实的多个数据集以及超过 202,000 个事实和 814,000 个图像的大规模数据集中应用了所研究的方法。