Computing the marginal likelihood (ML) of a model requires marginalizing out all of the parameters and latent variables, a difficult high-dimensional summation or integration problem. To make matters worse, it is often hard to measure the accuracy of one's ML estimates. We present bidirectional Monte Carlo, a technique for obtaining accurate log-ML estimates on data simulated from a model. This method obtains stochastic lower bounds on the log-ML using annealed importance sampling or sequential Monte Carlo, and obtains stochastic upper bounds by running these same algorithms in reverse starting from an exact posterior sample. The true value can be sandwiched between these two stochastic bounds with high probability. Using the ground truth log-ML estimates obtained from our method, we quantitatively evaluate a wide variety of existing ML estimators on several latent variable models: clustering, a low rank approximation, and a binary attributes model. These experiments yield insights into how to accurately estimate marginal likelihoods.

提出一种名为双向蒙特卡罗的技术，利用混合重要性采样或序列蒙特卡罗方法获得模型的准确对数边缘似然估计的随机下限和随机上限，并使用该方法获得的对数边缘似然估计真值对多种现有的最大似然估计器进行quantitative评估，并从中得出关于如何准确估计边缘似然估计的insights。

双向蒙特卡罗求解边缘似然