Although Large Language Models (LLMs) are effective in performing various NLP
tasks, they still struggle to handle tasks that require extensive, real-world
knowledge, especially when dealing with long-tail facts (facts related to
long-tail entities). This limitation highlights the need to supplement LLMs
with non-parametric knowledge. To address this issue, we analysed the effects
of different types of non-parametric knowledge, including textual passage and
knowledge graphs (KGs). Since LLMs have probably seen the majority of factual
question-answering datasets already, to facilitate our analysis, we proposed a
fully automatic pipeline for creating a benchmark that requires knowledge of
long-tail facts for answering the involved questions. Using this pipeline, we
introduce the LTGen benchmark. We evaluate state-of-the-art LLMs in different
knowledge settings using the proposed benchmark. Our experiments show that LLMs
alone struggle with answering these questions, especially when the long-tail
level is high or rich knowledge is required. Nonetheless, the performance of
the same models improved significantly when they were prompted with
non-parametric knowledge. We observed that, in most cases, prompting LLMs with
KG triples surpasses passage-based prompting using a state-of-the-art
retriever. In addition, while prompting LLMs with both KG triples and documents
does not consistently improve knowledge coverage, it can dramatically reduce
hallucinations in the generated content.

大型语言模型在各种自然语言处理任务中表现出色，但在处理需要广泛、现实世界知识的任务，特别是那些涉及长尾实体的任务时，仍然存在困难。为了解决这个问题，本研究分析了不同类型的非参数化知识对语言模型的影响，其中包括文本片段和知识图谱。通过创建一个需要长尾事实知识来回答问题的基准测试工具，我们评估了最先进的语言模型在不同知识环境下的表现。实验结果表明，单独使用语言模型来回答这些问题存在困难，特别是在需要大量长尾知识或丰富知识的情况下。然而，当为语言模型提供非参数化知识时，这些模型的性能显著提高。我们观察到，在大多数情况下，使用知识图谱三元组作为提示的语言模型表现优于使用最先进的检索器的段落提示。此外，虽然同时为语言模型提供知识图谱三元组和文档并不能始终改善知识覆盖率，但可以显著减少生成内容中的幻觉。

利用知识图谱为涉及长尾事实的问答任务提供大型语言模型的引导

Prompting Large Language Models with Knowledge Graphs for Question  Answering Involving Long-tail Facts

The present study proposes incorporating non-parametric knowledge into the
diffusion least-mean-squares algorithm in the framework of a maximum a
posteriori (MAP) estimation. The proposed algorithm leads to a robust
estimation of an unknown parameter vector in a group of cooperative estimators.
Utilizing kernel density estimation and buffering some intermediate
estimations, the prior distribution and conditional likelihood of the
parameters vector in each node are calculated. Pseudo Huber loss function is
used for designing the likelihood function. Also, an error thresholding
function is defined to reduce the computational overhead as well as more
relaxation against noise, which stops the update every time an error is less
than a predefined threshold. The performance of the proposed algorithm is
examined in the stationary and non-stationary scenarios in the presence of
Gaussian and non-Gaussian noise. Results show the robustness of the proposed
algorithm in the presence of different noise types.

本研究将非参数知识融入最小均方扩散算法的最大后验估计框架中，提出了一种能够在合作估计器组中鲁棒地估计未知参数向量的算法。通过利用核密度估计和缓冲一些中间估计量，计算每个节点中参数向量的先验分布和条件似然。设计了似然函数使用伪 Huber 损失函数，定义了一个误差阈值函数来减少计算开销并更好地抵抗噪声，当误差小于预定义阈值时停止更新。在存在高斯和非高斯噪声的稳态和非稳态情况下对所提出的算法的性能进行了检验。结果表明，该算法在不同类型噪声存在下具有鲁棒性。

自适应网络上的鲁棒非参数基于知识的扩散最小均方

Robust Non-parametric Knowledge-based Diffusion Least Mean Squares over  Adaptive Networks

Meta learning have achieved promising performance in low-resource text
classification which aims to identify target classes with knowledge transferred
from source classes with sets of small tasks named episodes. However, due to
the limited training data in the meta-learning scenario and the inherent
properties of parameterized neural networks, poor generalization performance
has become a pressing problem that needs to be addressed. To deal with this
issue, we propose a meta-learning based method called Retrieval-Augmented Meta
Learning(RAML). It not only uses parameterization for inference but also
retrieves non-parametric knowledge from an external corpus to make inferences,
which greatly alleviates the problem of poor generalization performance caused
by the lack of diverse training data in meta-learning. This method differs from
previous models that solely rely on parameters, as it explicitly emphasizes the
importance of non-parametric knowledge, aiming to strike a balance between
parameterized neural networks and non-parametric knowledge. The model is
required to determine which knowledge to access and utilize during inference.
Additionally, our multi-view passages fusion network module can effectively and
efficiently integrate the retrieved information into low-resource
classification task. The extensive experiments demonstrate that RAML
significantly outperforms current SOTA low-resource text classification models.

基于检索增强元学习 (RAML) 的元学习方法，通过从外部语料库中检索非参数性知识以进行推理，以解决元学习场景中缺乏多样化训练数据导致的泛化性能差的问题，并在低资源文本分类任务中显著优于当前最优模型。