The entropy bottleneck introduced by Ball\'e et al. is a common component
used in many learned compression models. It encodes a transformed latent
representation using a static distribution whose parameters are learned during
training. However, the actual distribution of the latent data may vary wildly
across different inputs. The static distribution attempts to encompass all
possible input distributions, thus fitting none of them particularly well. This
unfortunate phenomenon, sometimes known as the amortization gap, results in
suboptimal compression. To address this issue, we propose a method that
dynamically adapts the encoding distribution to match the latent data
distribution for a specific input. First, our model estimates a better encoding
distribution for a given input. This distribution is then compressed and
transmitted as an additional side-information bitstream. Finally, the decoder
reconstructs the encoding distribution and uses it to decompress the
corresponding latent data. Our method achieves a Bj{\o}ntegaard-Delta (BD)-rate
gain of -7.10% on the Kodak test dataset when applied to the standard
fully-factorized architecture. Furthermore, considering computational
complexity, the transform used by our method is an order of magnitude cheaper
in terms of Multiply-Accumulate (MAC) operations compared to related
side-information methods such as the scale hyperprior.

提出一种动态适应编码分布以匹配特定输入中的潜在数据分布的方法，通过估计更好的编码分布，并将其作为附加的辅助信息比特流进行压缩和传输，而后解码器使用该编码分布来解压对应的潜在数据。该方法在标准全分解架构上能够获得 - 7.10% 的 Bjøntegaard-Delta (BD)-rate 增益，并且在计算复杂性方面，与相关的辅助信息方法（如尺度超先验）相比，我们方法使用的变换在乘加（MAC）操作方面具有数量级上的更低成本。

编码分布的学习压缩

Learned Compression of Encoding Distributions

Inference models are a key component in scaling variational inference to deep
latent variable models, most notably as encoder networks in variational
auto-encoders (VAEs). By replacing conventional optimization-based inference
with a learned model, inference is amortized over data examples and therefore
more computationally efficient. However, standard inference models are
restricted to direct mappings from data to approximate posterior estimates. The
failure of these models to reach fully optimized approximate posterior
estimates results in an amortization gap. We aim toward closing this gap by
proposing iterative inference models, which learn to perform inference
optimization through repeatedly encoding gradients. Our approach generalizes
standard inference models in VAEs and provides insight into several empirical
findings, including top-down inference techniques. We demonstrate the inference
optimization capabilities of iterative inference models and show that they
outperform standard inference models on several benchmark data sets of images
and text.

我们提出了迭代推理模型，它通过反复编码梯度来学习执行推理优化，从而实现对标准推理模型在多个基准数据集的图像和文本上的超越。