Transformer-based approaches have achieved superior performance in image
restoration, since they can model long-term dependencies well. However, the
limitation in capturing local information restricts their capacity to remove
degradations. While existing approaches attempt to mitigate this issue by
incorporating convolutional operations, the core component in Transformer,
i.e., self-attention, which serves as a low-pass filter, could unintentionally
dilute or even eliminate the acquired local patterns. In this paper, we propose
HIT, a simple yet effective High-frequency Injected Transformer for image
restoration. Specifically, we design a window-wise injection module (WIM),
which incorporates abundant high-frequency details into the feature map, to
provide reliable references for restoring high-quality images. We also develop
a bidirectional interaction module (BIM) to aggregate features at different
scales using a mutually reinforced paradigm, resulting in spatially and
contextually improved representations. In addition, we introduce a spatial
enhancement unit (SEU) to preserve essential spatial relationships that may be
lost due to the computations carried out across channel dimensions in the BIM.
Extensive experiments on 9 tasks (real noise, real rain streak, raindrop,
motion blur, moir\'e, shadow, snow, haze, and low-light condition) demonstrate
that HIT with linear computational complexity performs favorably against the
state-of-the-art methods. The source code and pre-trained models will be
available at this https URL

我们提出了 HIT，一种简单而有效的高频注入变压器用于图像恢复，通过设计窗口注入模块（WIM），将丰富的高频细节融入特征图，为高质量图像恢复提供可靠的参考，并使用双向交互模块（BIM）以相互增强的方式汇总不同尺度的特征，从而改善空间上和背景上的表示。

观察再行动：高频注入变换器用于图像恢复

Look-Around Before You Leap: High-Frequency Injected Transformer for  Image Restoration

Current instruction-based editing methods, such as InstructPix2Pix, often
fail to produce satisfactory results in complex scenarios due to their
dependence on the simple CLIP text encoder in diffusion models. To rectify
this, this paper introduces SmartEdit, a novel approach to instruction-based
image editing that leverages Multimodal Large Language Models (MLLMs) to
enhance their understanding and reasoning capabilities. However, direct
integration of these elements still faces challenges in situations requiring
complex reasoning. To mitigate this, we propose a Bidirectional Interaction
Module that enables comprehensive bidirectional information interactions
between the input image and the MLLM output. During training, we initially
incorporate perception data to boost the perception and understanding
capabilities of diffusion models. Subsequently, we demonstrate that a small
amount of complex instruction editing data can effectively stimulate
SmartEdit's editing capabilities for more complex instructions. We further
construct a new evaluation dataset, Reason-Edit, specifically tailored for
complex instruction-based image editing. Both quantitative and qualitative
results on this evaluation dataset indicate that our SmartEdit surpasses
previous methods, paving the way for the practical application of complex
instruction-based image editing.

SmartEdit 是一种基于指令的图像编辑方法，利用多模态大型语言模型（MLLMS）增强理解和推理能力，并通过双向交互模块实现输入图像和 MLLM 输出之间的综合双向信息交互，从而实现更复杂指令下的图像编辑。通过新构建的 Reason-Edit 评估数据集，定量和定性结果表明 SmartEdit 优于先前的方法，为复杂指令 - based 图像编辑的实际应用铺平了道路。