Large Language Models (LLMs) encode meanings of words in the form of
distributed semantics. Distributed semantics capture common statistical
patterns among language tokens (words, phrases, and sentences) from large
amounts of data. LLMs perform exceedingly well across General Language
Understanding Evaluation (GLUE) tasks designed to test a model's understanding
of the meanings of the input tokens. However, recent studies have shown that
LLMs tend to generate unintended, inconsistent, or wrong texts as outputs when
processing inputs that were seen rarely during training, or inputs that are
associated with diverse contexts (e.g., well-known hallucination phenomenon in
language generation tasks). Crowdsourced and expert-curated knowledge graphs
such as ConceptNet are designed to capture the meaning of words from a compact
set of well-defined contexts. Thus LLMs may benefit from leveraging such
knowledge contexts to reduce inconsistencies in outputs. We propose a novel
ensemble learning method, Interpretable Ensemble Representation Learning
(IERL), that systematically combines LLM and crowdsourced knowledge
representations of input tokens. IERL has the distinct advantage of being
interpretable by design (when was the LLM context used vs. when was the
knowledge context used?) over state-of-the-art (SOTA) methods, allowing
scrutiny of the inputs in conjunction with the parameters of the model,
facilitating the analysis of models' inconsistent or irrelevant outputs.
Although IERL is agnostic to the choice of LLM and crowdsourced knowledge, we
demonstrate our approach using BERT and ConceptNet. We report improved or
competitive results with IERL across GLUE tasks over current SOTA methods and
significantly enhanced model interpretability.

研究通过使用知识图谱来优化大型语言模型的性能，并提出了一种能够解释的集成学习方法 (IERL)，在 General Language Understanding Evaluation (GLUE) 任务中表现良好。

IERL: 可解释的集成表示学习 —— 结合众包知识和分布式语义表示

IERL: Interpretable Ensemble Representation Learning -- Combining  CrowdSourced Knowledge and Distributed Semantic Representations

We propose a nonmonotonic Description Logic of typicality able to account for
the phenomenon of concept combination of prototypical concepts. The proposed
logic relies on the logic of typicality ALC TR, whose semantics is based on the
notion of rational closure, as well as on the distributed semantics of
probabilistic Description Logics, and is equipped with a cognitive heuristic
used by humans for concept composition. We first extend the logic of typicality
ALC TR by typicality inclusions whose intuitive meaning is that "there is
probability p about the fact that typical Cs are Ds". As in the distributed
semantics, we define different scenarios containing only some typicality
inclusions, each one having a suitable probability. We then focus on those
scenarios whose probabilities belong to a given and fixed range, and we exploit
such scenarios in order to ascribe typical properties to a concept C obtained
as the combination of two prototypical concepts. We also show that reasoning in
the proposed Description Logic is EXPTIME-complete as for the underlying ALC.

本文提出了一种能够描述原型概念组合现象的非单调典型性描述逻辑，它基于合理封闭的典型性描述逻辑 ALC TR 和概率描述逻辑的分布式语义，使用了人类用于概念组合的认知启发式。在此基础上，我们还给出了该逻辑的 EXPTIME 完全性推理方法。

一个基于描述逻辑的常识概念组合框架：整合典型性、概率和认知启发

A Description Logic Framework for Commonsense Conceptual Combination  Integrating Typicality, Probabilities and Cognitive Heuristics

This paper investigates how linguistic knowledge mined from large text
corpora can aid the generation of natural language descriptions of videos.
Specifically, we integrate both a neural language model and distributional
semantics trained on large text corpora into a recent LSTM-based architecture
for video description. We evaluate our approach on a collection of Youtube
videos as well as two large movie description datasets showing significant
improvements in grammaticality while modestly improving descriptive quality.

本文探讨了如何从大型文本语料库中挖掘语言知识以帮助生成视频的自然语言描述，并将神经语言模型和分布式语义训练应用于最近的基于 LSTM 的视频描述体系结构。我们在一组 Youtube 视频和两个大型电影描述数据集上评估了我们的方法，在改善语法正确性的同时适度提高了描述质量。