Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of
Large Language Models (LLMs) to various downstream applications. However, the
effectiveness of the PEFT diminishes notably when downstream tasks require
accurate learning of factual knowledge. In this paper, we adopt a semantic
perspective to investigate this phenomenon, uncovering the reasons behind
PEFT's limitations in knowledge learning task. Our findings reveal that: (1)
PEFT presents a notable risk of pushing the model away from the intended
knowledge target; (2) multiple knowledge interfere with each other, and such
interference suppresses the learning and expression of knowledge features.
Based on these insights, we introduce a data filtering strategy to exclude data
that is detrimental to knowledge learning and a re-weighted learning strategy
to make the model attentive to semantic distance during knowledge learning.
Experimental results demonstrate the effectiveness of the proposed method on
open-source large language model, further validate the semantic challenge in
PEFT, thus paving the way for future research.

通过采用语义视角研究，揭示了 Parameter-Efficient Fine-Tuning（PEFT）方法在知识学习任务中的限制原因，并提出了数据过滤策略和语义距离加权学习策略，以提高模型对知识学习的关注度。实验证明了该方法在开源大语言模型上的有效性，并为未来的研究铺平道路。

语义是信标：揭示参数高效微调的语义视角在知识学习中的应用

Semantic are Beacons: A Semantic Perspective for Unveiling  Parameter-Efficient Fine-Tuning in Knowledge Learning

In the swiftly advancing realm of communication technologies, Semantic
Communication (SemCom), which emphasizes knowledge understanding and
processing, has emerged as a hot topic. By integrating artificial intelligence
technologies, SemCom facilitates a profound understanding, analysis and
transmission of communication content. In this chapter, we clarify the means of
knowledge learning in SemCom with a particular focus on the utilization of
Knowledge Graphs (KGs). Specifically, we first review existing efforts that
combine SemCom with knowledge learning. Subsequently, we introduce a
KG-enhanced SemCom system, wherein the receiver is carefully calibrated to
leverage knowledge from its static knowledge base for ameliorating the decoding
performance. Contingent upon this framework, we further explore potential
approaches that can empower the system to operate in evolving knowledge base
more effectively. Furthermore, we investigate the possibility of integration
with Large Language Models (LLMs) for data augmentation, offering additional
perspective into the potential implementation means of SemCom. Extensive
numerical results demonstrate that the proposed framework yields superior
performance on top of the KG-enhanced decoding and manifests its versatility
under different scenarios.

通过集成人工智能技术，本文介绍了以知识理解和处理为重点的语义通信（SemCom）在知识学习方面的应用，尤其关注知识图谱（KGs）的利用，提出了增强型 KGs 的 SemCom 系统，并探索了在不断演化的知识库中更有效地操作的潜在方法和与大型语言模型（LLMs）进行数据增强的可能性。广泛的数值结果表明，所提出的框架在 KG 增强的解码上具有卓越的性能并展示了其在不同场景下的多功能性。

语义沟通与知识学习的相互作用

Interplay of Semantic Communication and Knowledge Learning

Big data have the characteristics of enormous volume, high velocity,
diversity, value-sparsity, and uncertainty, which lead the knowledge learning
from them full of challenges. With the emergence of crowdsourcing, versatile
information can be obtained on-demand so that the wisdom of crowds is easily
involved to facilitate the knowledge learning process. During the past thirteen
years, researchers in the AI community made great efforts to remove the
obstacles in the field of learning from crowds. This concentrated survey paper
comprehensively reviews the technical progress in crowdsourcing learning from a
systematic perspective that includes three dimensions of data, models, and
learning processes. In addition to reviewing existing important work, the paper
places a particular emphasis on providing some promising blueprints on each
dimension as well as discussing the lessons learned from our past research
work, which will light up the way for new researchers and encourage them to
pursue new contributions.

本文从数据、模型、学习三个维度，全面回顾了 13 年来 AI 社区在众包学习领域的进展，着重提出了每个维度的一些有前途的蓝图，并探讨了过去研究的经验教训，旨在为新研究者提供指引，鼓励他们做出新的贡献。