In reality, images often exhibit multiple degradations, such as rain and fog
at night (triple degradations). However, in many cases, individuals may not
want to remove all degradations, for instance, a blurry lens revealing a
beautiful snowy landscape (double degradations). In such scenarios, people may
only desire to deblur. These situations and requirements shed light on a new
challenge in image restoration, where a model must perceive and remove specific
degradation types specified by human commands in images with multiple
degradations. We term this task Referring Flexible Image Restoration (RFIR). To
address this, we first construct a large-scale synthetic dataset called RFIR,
comprising 153,423 samples with the degraded image, text prompt for specific
degradation removal and restored image. RFIR consists of five basic degradation
types: blur, rain, haze, low light and snow while six main sub-categories are
included for varying degrees of degradation removal. To tackle the challenge,
we propose a novel transformer-based multi-task model named TransRFIR, which
simultaneously perceives degradation types in the degraded image and removes
specific degradation upon text prompt. TransRFIR is based on two devised
attention modules, Multi-Head Agent Self-Attention (MHASA) and Multi-Head Agent
Cross Attention (MHACA), where MHASA and MHACA introduce the agent token and
reach the linear complexity, achieving lower computation cost than vanilla
self-attention and cross-attention and obtaining competitive performances. Our
TransRFIR achieves state-of-the-art performances compared with other
counterparts and is proven as an effective architecture for image restoration.
We release our project at this https URL

在图像修复中，人们常常希望根据指令去除特定的图像退化现象，本文提出了一种名为 TransRFIR 的新型建模方法，它通过感知图像中的不同退化类型并根据文本提示去除特定的退化，该方法基于两个设计的注意力模块，即 MHASA 和 MHACA，具有较低的计算成本和优秀的性能，可以作为图像修复的一种有效架构。