This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs are widely used in various fields in applications such as generative modeling. However, a unified understanding of these concepts has still been unexplored. In this paper, we show that the KL-divergence and the IPMs can be represented as maximal likelihoods differing only by sampling schemes, and use this result to derive a unified form of the IPMs and a relaxed estimation method. To develop the estimation problem, we construct an unconstrained maximum likelihood estimator to perform DRE with a stratified sampling scheme. We further propose a novel class of probability divergences, called the Density Ratio Metrics (DRMs), that interpolates the KL-divergence and the IPMs. In addition to these findings, we also introduce some applications of the DRMs, such as DRE and generative adversarial networks. In experiments, we validate the effectiveness of our proposed methods.

本文从最大似然密度比估计的角度提供了一个统一的视角，用于解释Kullback-Leibler（KL）散度和积分概率度量（IPMs）。我们表明KL散度和IPMs可以表示为仅有样本采样方案不同的最大似然估计值的形式，利用这个结果导出了IPMs的统一形式和一种松弛的估计方法。我们构建了一个无限制最大似然估计器来执行分层采样方案的DRE，进一步提出了一个新的概率差异类，称为密度比度量（DRMs），其插值了KL散度和IPMs。此外，我们还介绍了一些DRMs的应用，例如DRE和GAN。在实验中，我们验证了我们提出的方法的有效性。

Unified Perspective on Probability Divergence via Maximum Likelihood
  Density Ratio Estimation: Bridging KL-Divergence and Integral Probability
  Metrics

通过最大似然密度比估计统一考虑概率差异：连接KL-差异和积分概率度量