Deep latent variable models have seen recent success in many data domains.
Lossless compression is an application of these models which, despite having
the potential to be highly useful, has yet to be implemented in a practical
manner. We present `Bits Back with ANS' (BB-ANS), a scheme to perform lossless
compression with latent variable models at a near optimal rate. We demonstrate
this scheme by using it to compress the MNIST dataset with a variational
auto-encoder model (VAE), achieving compression rates superior to standard
methods with only a simple VAE. Given that the scheme is highly amenable to
parallelization, we conclude that with a sufficiently high quality generative
model this scheme could be used to achieve substantial improvements in
compression rate with acceptable running time. We make our implementation
available open source at this https URL .

我们提出了一种名为 BB-ANS 的方案，用于在潜在变量模型中进行无损压缩，通过使用变分自动编码器模型来压缩 MNIST 数据集，实现了比标准方法更高的压缩率。

使用 Bits Back 编码和潜在变量进行实用的无损压缩

Practical Lossless Compression with Latent Variables using Bits Back  Coding

This paper is concerned with algorithms for prediction of discrete sequences
over a finite alphabet, using variable order Markov models. The class of such
algorithms is large and in principle includes any lossless compression
algorithm. We focus on six prominent prediction algorithms, including Context
Tree Weighting (CTW), Prediction by Partial Match (PPM) and Probabilistic
Suffix Trees (PSTs). We discuss the properties of these algorithms and compare
their performance using real life sequences from three domains: proteins,
English text and music pieces. The comparison is made with respect to
prediction quality as measured by the average log-loss. We also compare
classification algorithms based on these predictors with respect to a number of
large protein classification tasks. Our results indicate that a "decomposed"
CTW (a variant of the CTW algorithm) and PPM outperform all other algorithms in
sequence prediction tasks. Somewhat surprisingly, a different algorithm, which
is a modification of the Lempel-Ziv compression algorithm, significantly
outperforms all algorithms on the protein classification problems.

本研究探究了使用变量阶马尔可夫模型进行离散序列预测的算法，并比较了六种著名的预测算法和它们在三个不同领域的表现情况。研究结果表明，在序列预测任务中，一种名为 “decomposed” CTW 的变种和 PPM 算法表现最佳，而在蛋白质分类问题中一种名为 Lempel-Ziv 压缩算法的修改版显著胜出。