Deep learning-based joint source-channel coding (deep JSCC) has been
demonstrated as an effective approach for wireless image transmission.
Nevertheless, current research has concentrated on minimizing a standard
distortion metric such as Mean Squared Error (MSE), which does not necessarily
improve the perceptual quality. To address this issue, we propose DiffJSCC, a
novel framework that leverages pre-trained text-to-image diffusion models to
enhance the realism of images transmitted over the channel. The proposed
DiffJSCC utilizes prior deep JSCC frameworks to deliver an initial
reconstructed image at the receiver. Then, the spatial and textual features are
extracted from the initial reconstruction, which, together with the channel
state information (e.g., signal-to-noise ratio, SNR), are passed to a control
module to fine-tune the pre-trained Stable Diffusion model. Extensive
experiments on the Kodak dataset reveal that our method significantly surpasses
both conventional methods and prior deep JSCC approaches on perceptual metrics
such as LPIPS and FID scores, especially with poor channel conditions and
limited bandwidth. Notably, DiffJSCC can achieve highly realistic
reconstructions for 768x512 pixel Kodak images with only 3072 symbols (<0.008
symbols per pixel) under 1dB SNR. Our code will be released in
this https URL

利用预训练的文本到图像扩散模型，提出了一种名为 DiffJSCC 的新框架，用于增强信道传输的图像的逼真度，通过利用空间和文本特征以及信道状态信息（如信噪比）对预先训练的稳定扩散模型进行微调，实验证明该方法在感知度量方面显著优于常规方法和以往的深度联合源信道编码方法，尤其在信道条件差和带宽有限的情况下表现出色，即使在 1dB 信噪比下，DiffJSCC 也能以每像素不到 0.008 个符号的速度对 768x512 像素的 Kodak 图像进行高度逼真的重建。

基于扩散辅助的高逼真无线图像传输的联合源信道编码

Diffusion-Aided Joint Source Channel Coding For High Realism Wireless  Image Transmission

We consider the image transmission problem over a noisy wireless channel via
deep learning-based joint source-channel coding (DeepJSCC) along with a
denoising diffusion probabilistic model (DDPM) at the receiver. Specifically,
we are interested in the perception-distortion trade-off in the practical
finite block length regime, in which separate source and channel coding can be
highly suboptimal. We introduce a novel scheme that utilizes the range-null
space decomposition of the target image. We transmit the range-space of the
image after encoding and employ DDPM to progressively refine its null space
contents. Through extensive experiments, we demonstrate significant
improvements in distortion and perceptual quality of reconstructed images
compared to standard DeepJSCC and the state-of-the-art generative
learning-based method. We will publicly share our source code to facilitate
further research and reproducibility.

通过深度学习的联合源信道编码（DeepJSCC）和接收端的去噪扩散概率模型（DDPM），我们考虑在噪声无线信道上的图像传输问题，并在实际有限块长度范围内关注感知失真权衡，其中独立的源编码和信道编码可能高度亚最优。我们引入了一种利用目标图像的范围 - 零空间分解的新方案。我们经过编码传输图像的范围空间，并采用 DDPM 逐步改进其零空间内容。通过大量实验，我们证明与标准 DeepJSCC 和最先进的生成学习方法相比，重建图像的失真和感知质量有显着改进。我们将公开共享我们的源代码以促进进一步的研究和可重复性。