State inference and parameter learning in sequential models can be
successfully performed with approximation techniques that maximize the evidence
lower bound to the marginal log-likelihood of the data distribution. These
methods may be referred to as Dynamical Variational Autoencoders, and our
specific focus lies on the deep Kalman filter. It has been shown that the ELBO
objective can oversimplify data representations, potentially compromising
estimation quality. Tighter Monte Carlo objectives have been proposed in the
literature to enhance generative modeling performance. For instance, the IWAE
objective uses importance weights to reduce the variance of marginal
log-likelihood estimates. In this paper, importance sampling is applied to the
DKF framework for learning deep Markov models, resulting in the IW-DKF, which
shows an improvement in terms of log-likelihood estimates and KL divergence
between the variational distribution and the transition model. The framework
using the sampled DKF update rule is also accommodated to address sequential
state and parameter estimation when working with highly non-linear
physics-based models. An experiment with the 3-space Lorenz attractor shows an
enhanced generative modeling performance and also a decrease in RMSE when
estimating the model parameters and latent states, indicating that tighter MCOs
lead to improved state inference performance.

在这篇论文中，通过重要性采样将序贯模型与 DKF 框架相结合，得到了 IW-DKF 方法，通过降低边际对数似然估计的方差，改进了生成建模性能和状态推断性能。

关于采样对深度序贯状态估计的影响

On the Impact of Sampling on Deep Sequential State Estimation

Variational autoencoders (VAEs) are powerful deep generative models widely
used to represent high-dimensional complex data through a low-dimensional
latent space learned in an unsupervised manner. In the original VAE model, the
input data vectors are processed independently. Recently, a series of papers
have presented different extensions of the VAE to process sequential data,
which model not only the latent space but also the temporal dependencies within
a sequence of data vectors and corresponding latent vectors, relying on
recurrent neural networks or state-space models. In this paper, we perform a
literature review of these models. We introduce and discuss a general class of
models, called dynamical variational autoencoders (DVAEs), which encompasses a
large subset of these temporal VAE extensions. Then, we present in detail seven
recently proposed DVAE models, with an aim to homogenize the notations and
presentation lines, as well as to relate these models with existing classical
temporal models. We have reimplemented those seven DVAE models and present the
results of an experimental benchmark conducted on the speech
analysis-resynthesis task (the PyTorch code is made publicly available). The
paper concludes with a discussion on important issues concerning the DVAE class
of models and future research guidelines.

本文对可变自编码器 (VAE) 拓展至处理序列数据的方法进行了综述，提出并讨论了动态变分自编码器 (DVAEs) 这一类模型，详细介绍了七种 DVAE 模型，并通过语音分析 - 重构任务的实验基准进行了验证，最后探讨了 DVAE 模型的重要问题和未来的研究方向。