Large language models (LLM) are generating information at a rapid pace,
requiring users to increasingly rely and trust the data. Despite remarkable
advances of LLM, Information generated by LLM is not completely trustworthy,
due to challenges in information quality. Specifically, integrity of
Information quality decreases due to unreliable, biased, tokenization during
pre-training of LLM. Moreover, due to decreased information quality issues, has
led towards hallucination, fabricated information. Unreliable information can
lead towards flawed decisions in businesses, which impacts economic activity.
In this work, we introduce novel mathematical information quality evaluation of
LLM, we furthermore analyze and highlight information quality challenges,
scaling laws to systematically scale language models.

大型语言模型产生信息速度快，但信息质量不可信，由于预训练中的不可靠和有偏见的标记化导致信息质量下降，进而导致虚构和伪造信息。本文引入了一种新的数学信息质量评估方法，并分析了信息质量挑战和语言模型的可伸缩性规律。

走向可信的语言模型：大型语言模型信息质量的研究

Towards Trustable Language Models: Investigating Information Quality of  Large Language Models

This letter proposes a novel anti-interference technique, semantic
interference cancellation (SemantIC), for enhancing information quality towards
the sixth-generation (6G) wireless networks. SemantIC only requires the
receiver to concatenate the channel decoder with a semantic auto-encoder. This
constructs a turbo loop which iteratively and alternately eliminates noise in
the signal domain and the semantic domain. From the viewpoint of network
information theory, the neural network of the semantic auto-encoder stores side
information by training, and provides side information in iterative decoding,
as an implementation of the Wyner-Ziv theorem. Simulation results verify the
performance improvement by SemantIC without extra channel resource cost.

该论文提出了一种新颖的抗干扰技术 —— 语义干扰消除（SemantIC），用于提高第六代无线网络的信息质量。SemantIC 只需要接收器将信道译码器与语义自编码器级联，从网络信息理论的角度来看，语义自编码器的神经网络通过训练存储副信息，并在迭代解码中提供副信息，作为 Wyner-Ziv 定理的一种实现。仿真结果验证了 SemantIC 的性能改进，而不需要额外的信道资源成本。