The advent of Large Language Models (LLM) has revolutionized the field of
natural language processing, enabling significant progress in various
applications. One key area of interest is the construction of Knowledge Bases
(KB) using these powerful models. Knowledge bases serve as repositories of
structured information, facilitating information retrieval and inference tasks.
Our paper proposes LLM2KB, a system for constructing knowledge bases using
large language models, with a focus on the Llama 2 architecture and the
Wikipedia dataset. We perform parameter efficient instruction tuning for
Llama-2-13b-chat and StableBeluga-13B by training small injection models that
have only 0.05 % of the parameters of the base models using the Low Rank
Adaptation (LoRA) technique. These injection models have been trained with
prompts that are engineered to utilize Wikipedia page contexts of subject
entities fetched using a Dense Passage Retrieval (DPR) algorithm, to answer
relevant object entities for a given subject entity and relation. Our best
performing model achieved an average F1 score of 0.6185 across 21 relations in
the LM-KBC challenge held at the ISWC 2023 conference.

使用大型语言模型构建知识库的 LLM2KB 系统有不同于基础模型的参数紧凑的注入模型，通过 LoRA 技术调整指令以便使用 Wikipedia 页面上下文实体，并在 LM-KBC 挑战中取得了 0.6185 的平均 F1 得分。

LLM2KB：利用调整指导上下文感知的大型语言模型构建知识库

LLM2KB: Constructing Knowledge Bases using instruction tuned context  aware Large Language Models

In this paper, we propose a table and image generation task to verify how the
knowledge about entities acquired from natural language is retained in Vision &
Language (V & L) models. This task consists of two parts: the first is to
generate a table containing knowledge about an entity and its related image,
and the second is to generate an image from an entity with a caption and a
table containing related knowledge of the entity. In both tasks, the model must
know the entities used to perform the generation properly. We created the
Wikipedia Table and Image Generation (WikiTIG) dataset from about 200,000
infoboxes in English Wikipedia articles to perform the proposed tasks. We
evaluated the performance on the tasks with respect to the above research
question using the V & L model OFA, which has achieved state-of-the-art results
in multiple tasks. Experimental results show that OFA forgets part of its
entity knowledge by pre-training as a complement to improve the performance of
image related tasks.

本文提出了一个基于表格和图像生成的任务来验证自然语言中获取的实体知识如何在视觉和语言模型中得以保留，其包括两部分，一是生成包含有关实体及相关图像的表格，二是基于实体及其知识表生成相关图像。我们使用英文维基百科文章的 170,000 个信息框创建了维基百科表格和图像生成 (WikiTIG) 数据集并使用 OFA 模型对任务进行了评估，实验结果表明，OFA 在预训练过程中会忘记其中一部分实体知识。