Energy-based models (EBMs) are powerful probabilistic models, but suffer from
intractable sampling and density evaluation due to the partition function. As a
result, inference in EBMs relies on approximate sampling algorithms, leading to
a mismatch between the model and inference. Motivated by this, we consider the
sampler-induced distribution as the model of interest and maximize the
likelihood of this model. This yields a class of energy-inspired models (EIMs)
that incorporate learned energy functions while still providing exact samples
and tractable log-likelihood lower bounds. We describe and evaluate three
instantiations of such models based on truncated rejection sampling,
self-normalized importance sampling, and Hamiltonian importance sampling. These
models outperform or perform comparably to the recently proposed Learned
Accept/Reject Sampling algorithm and provide new insights on ranking Noise
Contrastive Estimation and Contrastive Predictive Coding. Moreover, EIMs allow
us to generalize a recent connection between multi-sample variational lower
bounds and auxiliary variable variational inference. We show how recent
variational bounds can be unified with EIMs as the variational family.

本文研究通过最大化随机抽样分布的似然函数从而获得一类能量启发式模型，这些模型包含了学习能量函数并提供精确样本和可计算的对数似然下界。同时，这些模型对比了不同的随机抽样算法并提供了对噪声对比估计和对比预测编码排名的新见解。