Recent progress in generative compression technology has significantly
improved the perceptual quality of compressed data. However, these advancements
primarily focus on producing high-frequency details, often overlooking the
ability of generative models to capture the prior distribution of image
content, thus impeding further bitrate reduction in extreme compression
scenarios (<0.05 bpp). Motivated by the capabilities of predictive language
models for lossless compression, this paper introduces a novel Unified Image
Generation-Compression (UIGC) paradigm, merging the processes of generation and
compression. A key feature of the UIGC framework is the adoption of
vector-quantized (VQ) image models for tokenization, alongside a multi-stage
transformer designed to exploit spatial contextual information for modeling the
prior distribution. As such, the dual-purpose framework effectively utilizes
the learned prior for entropy estimation and assists in the regeneration of
lost tokens. Extensive experiments demonstrate the superiority of the proposed
UIGC framework over existing codecs in perceptual quality and human perception,
particularly in ultra-low bitrate scenarios (<=0.03 bpp), pioneering a new
direction in generative compression.

本文介绍了一种新的统一图像生成 - 压缩（UIGC）范式，该范式将生成和压缩过程合并在一起，通过采用矢量量化图像模型和多阶段 Transformer 来利用空间上下文信息建模先验分布，从而在实现熵估计和丢失令牌再生成方面成功使用学习到的先验，实验结果表明该 UIGC 框架在感知质量和人类感知方面优于现有编解码器，在极低比特率场景下（<=0.03 bpp）开创了生成式压缩的新方向。

统一生成和压缩：多阶段变形器在超低比特率图像编码中的应用

Unifying Generation and Compression: Ultra-low bitrate Image Coding Via  Multi-stage Transformer

By optimizing the rate-distortion-realism trade-off, generative compression
approaches produce detailed, realistic images, even at low bit rates, instead
of the blurry reconstructions produced by rate-distortion optimized models.
However, previous methods do not explicitly control how much detail is
synthesized, which results in a common criticism of these methods: users might
be worried that a misleading reconstruction far from the input image is
generated. In this work, we alleviate these concerns by training a decoder that
can bridge the two regimes and navigate the distortion-realism trade-off. From
a single compressed representation, the receiver can decide to either
reconstruct a low mean squared error reconstruction that is close to the input,
a realistic reconstruction with high perceptual quality, or anything in
between. With our method, we set a new state-of-the-art in distortion-realism,
pushing the frontier of achievable distortion-realism pairs, i.e., our method
achieves better distortions at high realism and better realism at low
distortion than ever before.

通过优化失真、真实感和压缩率之间的平衡，生成式压缩方法可以在低比特率下产生详细，逼真的图像。本文提出了一种新方法，训练了一个解码器，能够在失真 - 真实感平衡中生成高质量的实际效果，创造了最新的 state-of-the-art 技术水平。

带有条件生成器的多现实图像压缩

Multi-Realism Image Compression with a Conditional Generator

Traditional image and video compression algorithms rely on hand-crafted
encoder/decoder pairs (codecs) that lack adaptability and are agnostic to the
data being compressed. Here we describe the concept of generative compression,
the compression of data using generative models, and suggest that it is a
direction worth pursuing to produce more accurate and visually pleasing
reconstructions at much deeper compression levels for both image and video
data. We also demonstrate that generative compression is orders-of-magnitude
more resilient to bit error rates (e.g. from noisy wireless channels) than
traditional variable-length coding schemes.

使用生成模型进行压缩可提高失真度和视觉效果，对视频和图像压缩有效。相比传统的可变长度编码方案，生成压缩的鲁棒性更高，适应于噪声无线信道。