Differentiable annealed importance sampling (DAIS), proposed by Geffner &
Domke (2021) and Zhang et al. (2021), allows optimizing, among others, over the
initial distribution of AIS. In this paper, we show that, in the limit of many
transitions, DAIS minimizes the symmetrized KL divergence (Jensen-Shannon
divergence) between the initial and target distribution. Thus, DAIS can be seen
as a form of variational inference (VI) in that its initial distribution is a
parametric fit to an intractable target distribution. We empirically evaluate
the usefulness of the initial distribution as a variational distribution on
synthetic and real-world data, observing that it often provides more accurate
uncertainty estimates than standard VI (optimizing the reverse KL divergence),
importance weighted VI, and Markovian score climbing (optimizing the forward KL
divergence).

DAIS 是一种可微分的持续退火重要性采样方法，它通过最小化初始和目标分布之间的对称 KL 散度来进行变分推断。通过实证评估其在合成和真实数据上作为变分分布的有用性，发现与标准 VI、重要性加权 VI 和马尔科夫分数爬升相比，它常常能提供更准确的不确定性估计。

可微退火重要采样最小化初始分布与目标分布之间的 Jensen-Shannon 散度

Differentiable Annealed Importance Sampling Minimizes The Jensen-Shannon  Divergence Between Initial and Target Distribution

Stochastic optimal control of dynamical systems is a crucial challenge in
sequential decision-making. Recently, control-as-inference approaches have had
considerable success, providing a viable risk-sensitive framework to address
the exploration-exploitation dilemma. Nonetheless, a majority of these
techniques only invoke the inference-control duality to derive a modified risk
objective that is then addressed within a reinforcement learning framework.
This paper introduces a novel perspective by framing risk-sensitive stochastic
control as Markovian score climbing under samples drawn from a conditional
particle filter. Our approach, while purely inference-centric, provides
asymptotically unbiased estimates for gradient-based policy optimization with
optimal importance weighting and no explicit value function learning. To
validate our methodology, we apply it to the task of learning neural
non-Gaussian feedback policies, showcasing its efficacy on numerical benchmarks
of stochastic dynamical systems.

该论文介绍了一种新颖的方法，通过从条件粒子滤波器中抽取样本，将风险敏感的随机控制看作马尔科夫评分上升问题，提供了渐进无偏估计的梯度优化策略，避免了显式值函数学习，通过在随机动态系统的数值基准测试中展示其有效性，应用于学习神经非高斯反馈策略。

风险敏感随机最优控制的 Rao-Blackwellized Markovian Score Climbing

Risk-Sensitive Stochastic Optimal Control as Rao-Blackwellized Markovian  Score Climbing

Modern variational inference (VI) uses stochastic gradients to avoid
intractable expectations, enabling large-scale probabilistic inference in
complex models. VI posits a family of approximating distributions q and then
finds the member of that family that is closest to the exact posterior p.
Traditionally, VI algorithms minimize the "exclusive Kullback-Leibler (KL)"
KL(q || p), often for computational convenience. Recent research, however, has
also focused on the "inclusive KL" KL(p || q), which has good statistical
properties that makes it more appropriate for certain inference problems. This
paper develops a simple algorithm for reliably minimizing the inclusive KL
using stochastic gradients with vanishing bias. This method, which we call
Markovian score climbing (MSC), converges to a local optimum of the inclusive
KL. It does not suffer from the systematic errors inherent in existing methods,
such as Reweighted Wake-Sleep and Neural Adaptive Sequential Monte Carlo, which
lead to bias in their final estimates. We illustrate convergence on a toy model
and demonstrate the utility of MSC on Bayesian probit regression for
classification as well as a stochastic volatility model for financial data.

本文介绍了一种可靠的使用随机梯度下降最小化包含 KL 散度的简单算法 Markovian score climbing (MSC)，它以不偏的方式收敛于包含 KL。对贝叶斯 probit 回归进行分类以及针对金融数据的随机波动率模型演示了 MSC 的效用。