Recent progress in generative compression technology has significantly
improved the perceptual quality of compressed data. However, these advancements
primarily focus on producing high-frequency details, often overlooking the
ability of generative models to capture the prior distribution of image
content, thus impeding further bitrate reduction in extreme compression
scenarios (<0.05 bpp). Motivated by the capabilities of predictive language
models for lossless compression, this paper introduces a novel Unified Image
Generation-Compression (UIGC) paradigm, merging the processes of generation and
compression. A key feature of the UIGC framework is the adoption of
vector-quantized (VQ) image models for tokenization, alongside a multi-stage
transformer designed to exploit spatial contextual information for modeling the
prior distribution. As such, the dual-purpose framework effectively utilizes
the learned prior for entropy estimation and assists in the regeneration of
lost tokens. Extensive experiments demonstrate the superiority of the proposed
UIGC framework over existing codecs in perceptual quality and human perception,
particularly in ultra-low bitrate scenarios (<=0.03 bpp), pioneering a new
direction in generative compression.

本文介绍了一种新的统一图像生成 - 压缩（UIGC）范式，该范式将生成和压缩过程合并在一起，通过采用矢量量化图像模型和多阶段 Transformer 来利用空间上下文信息建模先验分布，从而在实现熵估计和丢失令牌再生成方面成功使用学习到的先验，实验结果表明该 UIGC 框架在感知质量和人类感知方面优于现有编解码器，在极低比特率场景下（<=0.03 bpp）开创了生成式压缩的新方向。

统一生成和压缩：多阶段变形器在超低比特率图像编码中的应用

Unifying Generation and Compression: Ultra-low bitrate Image Coding Via  Multi-stage Transformer

Whilst diffusion probabilistic models can generate high quality image
content, key limitations remain in terms of both generating high-resolution
imagery and their associated high computational requirements. Recent
Vector-Quantized image models have overcome this limitation of image resolution
but are prohibitively slow and unidirectional as they generate tokens via
element-wise autoregressive sampling from the prior. By contrast, in this paper
we propose a novel discrete diffusion probabilistic model prior which enables
parallel prediction of Vector-Quantized tokens by using an unconstrained
Transformer architecture as the backbone. During training, tokens are randomly
masked in an order-agnostic manner and the Transformer learns to predict the
original tokens. This parallelism of Vector-Quantized token prediction in turn
facilitates unconditional generation of globally consistent high-resolution and
diverse imagery at a fraction of the computational expense. In this manner, we
can generate image resolutions exceeding that of the original training set
samples whilst additionally provisioning per-image likelihood estimates (in a
departure from generative adversarial approaches). Our approach achieves
state-of-the-art results in terms of Density (LSUN Bedroom: 1.51; LSUN
Churches: 1.12; FFHQ: 1.20) and Coverage (LSUN Bedroom: 0.83; LSUN Churches:
0.73; FFHQ: 0.80), and performs competitively on FID (LSUN Bedroom: 3.64; LSUN
Churches: 4.07; FFHQ: 6.11) whilst offering advantages in terms of both
computation and reduced training set requirements.

本文提出了一种新的离散扩散概率模型优先级，通过使用无约束的 Transformer 架构作为骨干结构实现了向量量化标记的并行预测，从而促进了图像生成的全局一致性和多样性。

利用离散吸收扩散实现并行令牌预测：从向量量化编码中快速生成高分辨率图像的 Transformer 解放

Unleashing Transformers: Parallel Token Prediction with Discrete  Absorbing Diffusion for Fast High-Resolution Image Generation from  Vector-Quantized Codes