Shannon Information theory has achieved great success in not only
communication technology where it was originally developed for but also many
other science and engineering fields such as machine learning and artificial
intelligence. Inspired by the famous weighting scheme TF-IDF, we discovered
that information entropy has a natural dual. We complement the classical
Shannon information theory by proposing a novel quantity, namely troenpy.
Troenpy measures the certainty, commonness and similarity of the underlying
distribution. To demonstrate its usefulness, we propose a troenpy based
weighting scheme for document with class labels, namely positive class
frequency (PCF). On a collection of public datasets we show the PCF based
weighting scheme outperforms the classical TF-IDF and a popular Optimal
Transportation based word moving distance algorithm in a kNN setting. We
further developed a new odds-ratio type feature, namely Expected Class
Information Bias(ECIB), which can be regarded as the expected odds ratio of the
information quantity entropy and troenpy. In the experiments we observe that
including the new ECIB features and simple binary term features in a simple
logistic regression model can further significantly improve the performance.
The simple new weighting scheme and ECIB features are very effective and can be
computed with linear order complexity.

提出一种基于信息熵的新量度 troenpy 并给出其用于文本分类的权重方案 PCF；提出了期望类信息偏差 (ECIB) 特征，可以用于逻辑回归模型中，实验结果表明这些新方法能显著提高性能。

一种新的 Shannon 信息对偶和加权方案

A Novel Dual of Shannon Information and Weighting Scheme

We define a notion of stable and measurable map between cones endowed with
measurability tests and show that it forms a cpo-enriched cartesian closed
category. This category gives a denotational model of an extension of PCF
supporting the main primitives of probabilistic functional programming, like
continuous and discrete probabilistic distributions, sampling, conditioning and
full recursion. We prove the soundness and adequacy of this model with respect
to a call-by-name operational semantics and give some examples of its
denotations.

定义了稳定可测地图的概念，并展示其构成了用可测测试测度的圆锥的 cpo-enriched 笛卡尔闭范畴，该范畴为支持概率函数编程的 PCF 扩展提供了一种指称模型，例如连续和离散概率分布，采样，条件和全递归。我们证明了该模型相对于名称调用操作语义的正确性和适用性，并且给出了一些其指称的示例。