Pre-trained models with large-scale training data, such as CLIP and Stable
Diffusion, have demonstrated remarkable performance in various high-level
computer vision tasks such as image understanding and generation from language
descriptions. Yet, their potential for low-level tasks such as image
restoration remains relatively unexplored. In this paper, we explore such
models to enhance image restoration. As off-the-shelf features (OSF) from
pre-trained models do not directly serve image restoration, we propose to learn
an additional lightweight module called Pre-Train-Guided Refinement Module
(PTG-RM) to refine restoration results of a target restoration network with
OSF. PTG-RM consists of two components, Pre-Train-Guided Spatial-Varying
Enhancement (PTG-SVE), and Pre-Train-Guided Channel-Spatial Attention
(PTG-CSA). PTG-SVE enables optimal short- and long-range neural operations,
while PTG-CSA enhances spatial-channel attention for restoration-related
learning. Extensive experiments demonstrate that PTG-RM, with its compact size
($<$1M parameters), effectively enhances restoration performance of various
models across different tasks, including low-light enhancement, deraining,
deblurring, and denoising.

使用大规模训练数据的预训练模型在各种高层次的计算机视觉任务中表现出色，但在低层次的图像修复任务中的潜力仍未得到充分探索。本文通过引入一个名为 Pre-Train-Guided Refinement Module (PTG-RM) 的轻量级模块，使用预训练模型的特征优化目标图像修复网络的修复结果，并通过各种实验验证了其在降噪、去雨、去模糊和低光照增强等任务中有效提升修复性能。