Large language models appear to learn facts from the large text corpora they
are trained on. Such facts are encoded implicitly within their many parameters,
making it difficult to verify or manipulate what knowledge has been learned.
Language models have recently been extended to multilingual language models
(MLLMs), enabling knowledge to be learned across hundreds of languages.
Meanwhile, knowledge graphs contain facts in an explicit triple format, which
require careful and costly curation and are only available in a few
high-resource languages, restricting their research and application. To address
these issues, we propose to enhance MLLMs with knowledge from multilingual
knowledge graphs (MLKGs) so as to tackle language and knowledge graph tasks
across many languages, including low-resource ones. Specifically, we introduce
a lightweight adapter set to enhance MLLMs with cross-lingual entity alignment
and facts from MLKGs for many languages. Experiments on common benchmarks show
that such enhancement benefits both MLLMs and MLKGs, achieving: (1) comparable
or improved performance for knowledge graph completion and entity alignment
relative to baselines, especially for low-resource languages (for which
knowledge graphs are unavailable); and (2) improved MLLM performance on
language understanding tasks that require multilingual factual knowledge; all
while maintaining performance on other general language tasks.

本文提出了在多种语言中增强多语言知识图谱 (MLKGs) 的多语言语言模型 (MLLMs) 的轻量级适配器，以利用跨语言实体对齐和从 MLKGs 中获取事实，并在共同基准实验中展示了该增强模型在语言理解任务和知识图谱任务方面的性能优势。

提高多语言知识和文本建模能力的适配器

Adapters for Enhanced Modeling of Multilingual Knowledge and Text

Multilingual knowledge graphs (KGs), such as YAGO and DBpedia, represent
entities in different languages. The task of cross-lingual entity alignment is
to match entities in a source language with their counterparts in target
languages. In this work, we investigate embedding-based approaches to encode
entities from multilingual KGs into the same vector space, where equivalent
entities are close to each other. Specifically, we apply graph convolutional
networks (GCNs) to combine multi-aspect information of entities, including
topological connections, relations, and attributes of entities, to learn entity
embeddings. To exploit the literal descriptions of entities expressed in
different languages, we propose two uses of a pretrained multilingual BERT
model to bridge cross-lingual gaps. We further propose two strategies to
integrate GCN-based and BERT-based modules to boost performance. Extensive
experiments on two benchmark datasets demonstrate that our method significantly
outperforms existing systems.

本文研究使用基于嵌入的方法将不同语言的实体编码到同一向量空间中，并提出使用预训练的多语言 BERT 模型来融合各个语言中的实体描述，以寻找源语言和目标语言之间相应实体的方法。结果表明，该方法在两个基准数据集上明显优于现有系统。