Estimating the mutual information from samples from a joint distribution is a
challenging problem in both science and engineering. In this work, we realize a
variational bound that generalizes both discriminative and generative
approaches. Using this bound, we propose a hybrid method to mitigate their
respective shortcomings. Further, we propose Predictive Quantization (PQ): a
simple generative method that can be easily combined with discriminative
estimators for minimal computational overhead. Our propositions yield a tighter
bound on the information thanks to the reduced variance of the estimator. We
test our methods on a challenging task of correlated high-dimensional Gaussian
distributions and a stochastic process involving a system of free particles
subjected to a fixed energy landscape. Empirical results show that hybrid
methods consistently improved mutual information estimates when compared to the
corresponding discriminative counterpart.

本研究提出了一种结合判别性和生成性方法的变分界的估算方法，我们进一步提出了一种名为 PQ 的简单生成方法，可以与判别性估计器轻松结合，可以在高维高斯分布和随机过程数据上实现更紧凑的互信息估计。