Shannon Information theory has achieved great success in not only
communication technology where it was originally developed for but also many
other science and engineering fields such as machine learning and artificial
intelligence. Inspired by the famous weighting scheme TF-IDF, we discovered
that information entropy has a natural dual. We complement the classical
Shannon information theory by proposing a novel quantity, namely troenpy.
Troenpy measures the certainty, commonness and similarity of the underlying
distribution. To demonstrate its usefulness, we propose a troenpy based
weighting scheme for document with class labels, namely positive class
frequency (PCF). On a collection of public datasets we show the PCF based
weighting scheme outperforms the classical TF-IDF and a popular Optimal
Transportation based word moving distance algorithm in a kNN setting. We
further developed a new odds-ratio type feature, namely Expected Class
Information Bias(ECIB), which can be regarded as the expected odds ratio of the
information quantity entropy and troenpy. In the experiments we observe that
including the new ECIB features and simple binary term features in a simple
logistic regression model can further significantly improve the performance.
The simple new weighting scheme and ECIB features are very effective and can be
computed with linear order complexity.

提出一种基于信息熵的新量度 troenpy 并给出其用于文本分类的权重方案 PCF；提出了期望类信息偏差 (ECIB) 特征，可以用于逻辑回归模型中，实验结果表明这些新方法能显著提高性能。

一种新的 Shannon 信息对偶和加权方案

A Novel Dual of Shannon Information and Weighting Scheme

Semantic communication, regarded as the breakthrough beyond the Shannon
paradigm, aims at the successful transmission of semantic information conveyed
by the source rather than the accurate reception of each single symbol or bit
regardless of its meaning. This article provides an overview on semantic
communications. After a brief review of Shannon information theory, we discuss
semantic communications with theory, framework, and system design enabled by
deep learning. Different from the symbol/bit error rate used for measuring
conventional communication systems, performance metrics for semantic
communications are also discussed. The article concludes with several open
questions in semantic communications.

该文论述了语义通信的概念和理论框架，重点探讨了基于深度学习的语义通信系统设计及其性能度量，最终提出若干该领域的待解决问题。

语义通信：原理与挑战

Semantic Communications: Principles and Challenges

Shannon information theory provides various measures of so-called "syntactic
information", which reflect the amount of statistical correlation between
systems. In contrast, the concept of "semantic information" refers to those
correlations which carry significance or "meaning" for a given system. Semantic
information plays an important role in many fields, including biology,
cognitive science, and philosophy, and there has been a long-standing interest
in formulating a broadly applicable and formal theory of semantic information.
In this paper we introduce such a theory. We define semantic information as the
syntactic information that a physical system has about its environment which is
causally necessary for the system to maintain its own existence. "Causal
necessity" is defined in terms of counter-factual interventions which scramble
correlations between the system and its environment, while "maintaining
existence" is defined in terms of the system's ability to keep itself in a low
entropy state. We also use recent results in nonequilibrium statistical physics
to analyze semantic information from a thermodynamic point of view. Our
framework is grounded in the intrinsic dynamics of a system coupled to an
environment, and is applicable to any physical system, living or otherwise. It
leads to formal definitions of several concepts that have been intuitively
understood to be related to semantic information, including "value of
information", "semantic content", and "agency".

本论文引入了一种精确的语义信息理论，该理论将语义信息定义为物理系统对其环境的句法信息，这种关联对于系统维持其低熵状态的原因至关重要，同时分析了语义信息的热力学和价值相关概念