Measuring Mutual Information (MI) between high-dimensional, continuous,
random variables from observed samples has wide theoretical and practical
applications. Recent work, MINE (Belghazi et al. 2018), focused on estimating
tight variational lower bounds of MI using neural networks, but assumed
unlimited supply of samples to prevent overfitting. In real world applications,
data is not always available at a surplus. In this work, we focus on improving
data efficiency and propose a Data-Efficient MINE Estimator (DEMINE), by
developing a relaxed predictive MI lower bound that can be estimated at higher
data efficiency by orders of magnitudes. The predictive MI lower bound also
enables us to develop a new meta-learning approach using task augmentation,
Meta-DEMINE, to improve generalization of the network and further boost
estimation accuracy empirically. With improved data-efficiency, our estimators
enables statistical testing of dependency at practical dataset sizes. We
demonstrate the effectiveness of our estimators on synthetic benchmarks and a
real world fMRI data, with application of inter-subject correlation analysis.

本文提出了一种基于神经网络的数据效率更高的 Mutual Information 估计方法 DEMINE，通过放松预测 MI 下限来提高数据效率，并采用任务扩充方法 Meta-DEMINE 进一步优化其推广性和估计准确性，可用于实际数据集大小的统计依赖性检测。