Large language models (LLMs) have demonstrated remarkable proficiency in
understanding and generating responses to complex queries through large-scale
pre-training. However, the efficacy of these models in memorizing and reasoning
among large-scale structured knowledge, especially world knowledge that
explicitly covers abundant factual information remains questionable. Addressing
this gap, our research investigates whether LLMs can effectively store, recall,
and reason with knowledge on a large scale comparable to latest knowledge bases
(KBs) such as Wikidata. Specifically, we focus on three crucial aspects to
study the viability: (1) the efficiency of LLMs with different sizes in
memorizing the exact knowledge in the large-scale KB; (2) the flexibility of
recalling the memorized knowledge in response to natural language queries; (3)
the capability to infer new knowledge through reasoning. Our findings indicate
that while LLMs hold promise as large-scale KBs capable of retrieving and
responding with flexibility, enhancements in their reasoning capabilities are
necessary to fully realize their potential.

大型语言模型在理解和生成复杂查询方面表现出了卓越的能力，但它们在存储、回忆和推理大规模结构化知识方面的有效性仍有待研究，本研究发现虽然大型语言模型有望作为大规模知识库灵活地检索和回答问题，但需要增强它们的推理能力以充分发挥潜力。

语言模型在规模上是否能充当知识库？

Can Language Models Act as Knowledge Bases at Scale?

Large Language Models are affected by the phenomena of memorizing and
forgetting their training data. But how do these vary by model size? We work
towards this question by investigating how the model size affects the model's
ability to discriminate a word's meaning in a given context. We introduce a
dataset called DeltaWords, which evaluates a model's ability to follow
instructions to select a sentence which replaces the target word with its
antonym. We show a weak inverse scaling trend, where task accuracy degrades as
model size increase, under extremely few-shot prompting regimes. We show that
increasing the number of examples tend to disproportionately benefit larger
models than smaller models.

研究了大型语言模型的记忆与遗忘现象及其与模型大小的关系，通过引入 DeltaWords 数据集证明在少量给定指令的情况下，随着模型大小的增加，任务准确率呈反比例降低趋势，但增加示例数量则更有利于大型模型。