Relative entropy coding (REC) algorithms encode a random sample following a
target distribution $Q$, using a coding distribution $P$ shared between the
sender and receiver. Sadly, general REC algorithms suffer from prohibitive
encoding times, at least on the order of $2^{D_{\text{KL}}[Q||P]}$, and faster
algorithms are limited to very specific settings. This work addresses this
issue by introducing a REC scheme utilizing space partitioning to reduce
runtime in practical scenarios. We provide theoretical analyses of our method
and demonstrate its effectiveness with both toy examples and practical
applications. Notably, our method successfully handles REC tasks with
$D_{\text{KL}}[Q||P]$ about three times what previous methods can manage and
reduces the compression rate by approximately 5-15\% in VAE-based lossless
compression on MNIST and INR-based lossy compression on CIFAR-10 compared to
previous methods, significantly improving the practicality of REC for neural
compression.

使用空间分割减少运行时间的相对熵编码（REC）算法可成功处理具有三倍于先前方法的 KL 散度，并将压缩率与先前方法相比提高了约 5-15％，显著改善了神经压缩中的 REC 方法的实用性。

加速相对熵编码的空间分割

Accelerating Relative Entropy Coding with Space Partitioning

Many common types of data can be represented as functions that map
coordinates to signal values, such as pixel locations to RGB values in the case
of an image. Based on this view, data can be compressed by overfitting a
compact neural network to its functional representation and then encoding the
network weights. However, most current solutions for this are inefficient, as
quantization to low-bit precision substantially degrades the reconstruction
quality. To address this issue, we propose overfitting variational Bayesian
neural networks to the data and compressing an approximate posterior weight
sample using relative entropy coding instead of quantizing and entropy coding
it. This strategy enables direct optimization of the rate-distortion
performance by minimizing the $\beta$-ELBO, and target different
rate-distortion trade-offs for a given network architecture by adjusting
$\beta$. Moreover, we introduce an iterative algorithm for learning prior
weight distributions and employ a progressive refinement process for the
variational posterior that significantly enhances performance. Experiments show
that our method achieves strong performance on image and audio compression
while retaining simplicity.

本文提出了一种基于变分贝叶斯神经网络的数据压缩方法，并使用相对熵编码代替量化和熵编码以优化 $eta$-ELBO 的速率失真性能，同时通过迭代算法学习先验权重分布并采用渐进式细化过程来有效提高压缩性能。实验证明，该方法在图像和音频压缩方面具有强大的性能，同时仍保持简洁。

基于贝叶斯隐式神经表达的压缩

Compression with Bayesian Implicit Neural Representations

Variational Autoencoders (VAEs) have seen widespread use in learned image
compression. They are used to learn expressive latent representations on which
downstream compression methods can operate with high efficiency. Recently
proposed 'bits-back' methods can indirectly encode the latent representation of
images with codelength close to the relative entropy between the latent
posterior and the prior. However, due to the underlying algorithm, these
methods can only be used for lossless compression, and they only achieve their
nominal efficiency when compressing multiple images simultaneously; they are
inefficient for compressing single images. As an alternative, we propose a
novel method, Relative Entropy Coding (REC), that can directly encode the
latent representation with codelength close to the relative entropy for single
images, supported by our empirical results obtained on the Cifar10, ImageNet32
and Kodak datasets. Moreover, unlike previous bits-back methods, REC is
immediately applicable to lossy compression, where it is competitive with the
state-of-the-art on the Kodak dataset.

提出了一种新的相对熵编码方法 Relative Entropy Coding (REC)，可应用于单图像的压缩，并在 Cifar10、ImageNet32 和 Kodak 数据集上进行了实证研究，得到了支持 REC 的实证结果；与之前的 bits-back 方法不同，REC 可立即应用于有损压缩，并在 Kodak 数据集上与最新技术相比具有竞争力。