In recent years, many convolutional neural network-based models are designed
for JPEG artifacts reduction, and have achieved notable progress. However, few
methods are suitable for extreme low-bitrate image compression artifacts
reduction. The main challenge is that the highly compressed image loses too
much information, resulting in reconstructing high-quality image difficultly.
To address this issue, we propose a multimodal fusion learning method for
text-guided JPEG artifacts reduction, in which the corresponding text
description not only provides the potential prior information of the highly
compressed image, but also serves as supplementary information to assist in
image deblocking. We fuse image features and text semantic features from the
global and local perspectives respectively, and design a contrastive loss built
upon contrastive learning to produce visually pleasing results. Extensive
experiments, including a user study, prove that our method can obtain better
deblocking results compared to the state-of-the-art methods.

本文提出一种基于多模态融合学习的文本引导下的 JPEG 压缩伪影降低方法，通过从全局和局部角度融合图像和文本语义特征，并基于对比学习构建对比损失函数，能够在极低码率下有效降低 JPEG 压缩伪影。经过实验证明，该方法能够获得比现有技术更好的降低伪影效果。