Transformer-based image denoising methods have achieved encouraging results
in the past year. However, it must uses linear operations to model long-range
dependencies, which greatly increases model inference time and consumes GPU
storage space. Compared with convolutional neural network-based methods,
current Transformer-based image denoising methods cannot achieve a balance
between performance improvement and resource consumption. In this paper, we
propose an Efficient Wavelet Transformer (EWT) for image denoising.
Specifically, we use Discrete Wavelet Transform (DWT) and Inverse Wavelet
Transform (IWT) for downsampling and upsampling, respectively. This method can
fully preserve the image features while reducing the image resolution, thereby
greatly reducing the device resource consumption of the Transformer model.
Furthermore, we propose a novel Dual-stream Feature Extraction Block (DFEB) to
extract image features at different levels, which can further reduce model
inference time and GPU memory usage. Experiments show that our method speeds up
the original Transformer by more than 80%, reduces GPU memory usage by more
than 60%, and achieves excellent denoising results. All code will be public.

提出了一种高效的小波变换器 (Efficient Wavelet Transformer，EWT) 方法，使用离散小波变换 (DWT) 和反小波变换 (IWT) 降采样和升采样以实现图像降噪。同时，还提出了一种新颖的双流特征提取块 (DFEB) 来在不同的级别提取图像特征，缩短模型推断时间和 GPU 存储器使用量。实验证明，该方法提高了原始 Transformer 的速度超过 80%，减少了 GPU 存储器使用量超过 60%，并取得了良好的去噪结果。