Variational flows allow practitioners to learn complex continuous
distributions, but approximating discrete distributions remains a challenge.
Current methodologies typically embed the discrete target in a continuous space
- usually via continuous relaxation or dequantization - and then apply a
continuous flow. These approaches involve a surrogate target that may not
capture the original discrete target, might have biased or unstable gradients,
and can create a difficult optimization problem. In this work, we develop a
variational flow family for discrete distributions without any continuous
embedding. First, we develop a measure-preserving and discrete (MAD) invertible
map that leaves the discrete target invariant, and then create a mixed
variational flow (MAD Mix) based on that map. We also develop an extension to
MAD Mix that handles joint discrete and continuous models. Our experiments
suggest that MAD Mix produces more reliable approximations than
continuous-embedding flows while being significantly faster to train.

开发了一种不需要连续嵌入的离散分布的变分流，称为 MAD Mix，通过测量保持和离散可逆映射来生成离散目标的可靠逼近，并且比连续嵌入的流训练速度显著更快。

混合变分流用于离散变量

Mixed Variational Flows for Discrete Variables

In this paper, we investigate the impact of numerical instability on the
reliability of sampling, density evaluation, and evidence lower bound (ELBO)
estimation in variational flows. We first empirically demonstrate that common
flows can exhibit a catastrophic accumulation of error: the numerical flow map
deviates significantly from the exact map -- which affects sampling -- and the
numerical inverse flow map does not accurately recover the initial input --
which affects density and ELBO computations. Surprisingly though, we find that
results produced by flows are often accurate enough for applications despite
the presence of serious numerical instability. In this work, we treat
variational flows as dynamical systems, and leverage shadowing theory to
elucidate this behavior via theoretical guarantees on the error of sampling,
density evaluation, and ELBO estimation. Finally, we develop and empirically
test a diagnostic procedure that can be used to validate results produced by
numerically unstable flows in practice.

本研究探讨了数值不稳定性对于变分流中的采样、密度评估和证据下界（ELBO）估计的可靠性的影响，发现虽然存在严重的数值不稳定性，但采样和计算结果仍然足够准确，并提出了一种诊断过程来验证实践中使用数值不稳定流的结果。