Deep learning-based joint source-channel coding (deep JSCC) has been
demonstrated as an effective approach for wireless image transmission.
Nevertheless, current research has concentrated on minimizing a standard
distortion metric such as Mean Squared Error (MSE), which does not necessarily
improve the perceptual quality. To address this issue, we propose DiffJSCC, a
novel framework that leverages pre-trained text-to-image diffusion models to
enhance the realism of images transmitted over the channel. The proposed
DiffJSCC utilizes prior deep JSCC frameworks to deliver an initial
reconstructed image at the receiver. Then, the spatial and textual features are
extracted from the initial reconstruction, which, together with the channel
state information (e.g., signal-to-noise ratio, SNR), are passed to a control
module to fine-tune the pre-trained Stable Diffusion model. Extensive
experiments on the Kodak dataset reveal that our method significantly surpasses
both conventional methods and prior deep JSCC approaches on perceptual metrics
such as LPIPS and FID scores, especially with poor channel conditions and
limited bandwidth. Notably, DiffJSCC can achieve highly realistic
reconstructions for 768x512 pixel Kodak images with only 3072 symbols (<0.008
symbols per pixel) under 1dB SNR. Our code will be released in
this https URL

利用预训练的文本到图像扩散模型，提出了一种名为 DiffJSCC 的新框架，用于增强信道传输的图像的逼真度，通过利用空间和文本特征以及信道状态信息（如信噪比）对预先训练的稳定扩散模型进行微调，实验证明该方法在感知度量方面显著优于常规方法和以往的深度联合源信道编码方法，尤其在信道条件差和带宽有限的情况下表现出色，即使在 1dB 信噪比下，DiffJSCC 也能以每像素不到 0.008 个符号的速度对 768x512 像素的 Kodak 图像进行高度逼真的重建。