Bayesian networks (BNs) are a foundational model in machine learning and
causal inference. Their graphical structure can handle high-dimensional
problems, divide-and-conquering them into a sparse collection of smaller ones;
underlies Judea Pearl's causality; and determines their explainability and
interpretability. Despite their popularity, there are few resources in the
literature on how to compute Shannon's entropy and the Kullback-Leibler (KL)
divergence for BNs under their most common distributional assumptions. In this
paper, we provide computationally efficient algorithms for both by leveraging
BNs' graphical structure, and we illustrate them with a complete set of
numerical examples. In the process, we show it is possible to reduce the
computational complexity of KL from cubic to quadratic for Gaussian BNs.

本研究提供了针对贝叶斯网络的计算机算法，通过利用网络的图形结构，将 Shannon 熵和 Kullback-Leibler 散度的计算复杂度从三次降低到二次，并提供了完整的数值示例。

贝叶斯网络的熵和库尔巴克 - 莱布勒散度：计算复杂度与高效实现

Entropy and the Kullback-Leibler Divergence for Bayesian Networks:  Computational Complexity and Efficient Implementation

Expert finding is an information retrieval task concerned with the search for
the most knowledgeable people, in some topic, with basis on documents
describing peoples activities. The task involves taking a user query as input
and returning a list of people sorted by their level of expertise regarding the
user query. This paper introduces a novel approach for combining multiple
estimators of expertise based on a multisensor data fusion framework together
with the Dempster-Shafer theory of evidence and Shannon's entropy. More
specifically, we defined three sensors which detect heterogeneous information
derived from the textual contents, from the graph structure of the citation
patterns for the community of experts, and from profile information about the
academic experts. Given the evidences collected, each sensor may define
different candidates as experts and consequently do not agree in a final
ranking decision. To deal with these conflicts, we applied the Dempster-Shafer
theory of evidence combined with Shannon's Entropy formula to fuse this
information and come up with a more accurate and reliable final ranking list.
Experiments made over two datasets of academic publications from the Computer
Science domain attest for the adequacy of the proposed approach over the
traditional state of the art approaches. We also made experiments against
representative supervised state of the art algorithms. Results revealed that
the proposed method achieved a similar performance when compared to these
supervised techniques, confirming the capabilities of the proposed framework.

该论文介绍了一种基于多传感器数据融合框架与极大熵及 Dempster-Shafer 证据理论相结合的新方法，用于解决查找某个主题下最有知识的人的信息检索任务。它应用于一个学术领域的两个数据集上，证明了该方法的广泛适用性和优越性，可以使它在性能上达到与传统的监督技术相同的水平。

利用香农熵的多传感器方法寻找学术专家

Finding Academic Experts on a MultiSensor Approach using Shannon's  Entropy

Understanding the complexity of human language requires an appropriate
analysis of the statistical distribution of words in texts. We consider the
information retrieval problem of detecting and ranking the relevant words of a
text by means of statistical information referring to the "spatial" use of the
words. Shannon's entropy of information is used as a tool for automatic keyword
extraction. By using The Origin of Species by Charles Darwin as a
representative text sample, we show the performance of our detector and compare
it with another proposals in the literature. The random shuffled text receives
special attention as a tool for calibrating the ranking indices.

本文通过统计文本中单词的空间使用情况，利用信息熵作为工具实现自动关键词提取，并以达尔文的《物种起源》为代表性文本样本进行性能测试和比较。